{"id":0,"url":"https:\/\/docs.atlan.com\/","title":"Atlan | Atlan Documentation","text":"Discover, trust, and govern your data & AI ecosystem Everything you need to get started with Atlan. ð Quick-start guide Step-by-step onboarding ð § Secure agent Enterprise-grade deployment options ð Playbooks automation Rule-based metadata updates at scale ð ¡ï¸ Govern & manage Create data contracts & policies ð Integrate Automation, collaboration & other integrations ï¸ Introductory walkthrough Play with APIs in minutes ð » Client SDKs Java, Python & more ð ¦ Packages Developer-built utilities and integrations Get started with Atlan by building the right strategy and setting a strong foundation."}
{"id":1,"url":"https:\/\/docs.atlan.com\/","title":"Atlan | Atlan Documentation","text":"A comprehensive look at Atlan's security philosophy, core values, and rigorous security procedures Help and support Find answers or contact our team for personalized assistance"}
{"id":2,"url":"https:\/\/developer.atlan.com\/","title":"Atlan - Developers","text":"Are you keen to activate your metadata? To automate tedious manual work, or reduce the complexity of some of your largest data challenges?"}
{"id":3,"url":"https:\/\/developer.atlan.com\/","title":"Atlan - Developers","text":"Welcome to our 🏡 for the developer community of Atlan. Here you'll find reference materials for working with Atlan programmatically."}
{"id":4,"url":"https:\/\/developer.atlan.com\/","title":"Atlan - Developers","text":"We look forward to helping you, the 🧑's of data, develop the future 🤖's of data!"}
{"id":5,"url":"https:\/\/docs.atlan.com\/#__docusaurus_skipToContent_fallback","title":"Atlan | Atlan Documentation","text":"Discover, trust, and govern your data & AI ecosystem Everything you need to get started with Atlan. ð Quick-start guide Step-by-step onboarding ð § Secure agent Enterprise-grade deployment options ð Playbooks automation Rule-based metadata updates at scale ð ¡ï¸ Govern & manage Create data contracts & policies ð Integrate Automation, collaboration & other integrations ï¸ Introductory walkthrough Play with APIs in minutes ð » Client SDKs Java, Python & more ð ¦ Packages Developer-built utilities and integrations Get started with Atlan by building the right strategy and setting a strong foundation."}
{"id":6,"url":"https:\/\/docs.atlan.com\/#__docusaurus_skipToContent_fallback","title":"Atlan | Atlan Documentation","text":"A comprehensive look at Atlan's security philosophy, core values, and rigorous security procedures Help and support Find answers or contact our team for personalized assistance"}
{"id":7,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":8,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":9,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":10,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":11,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":12,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":13,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":14,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":15,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":16,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":17,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":18,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":19,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":20,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":21,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":22,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":23,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":24,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":25,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":26,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":27,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":28,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":29,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":30,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":31,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":32,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":33,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":34,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":35,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":36,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":37,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":38,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":39,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":40,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":41,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":42,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":43,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":44,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":45,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":46,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":47,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":48,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":49,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":50,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":51,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":52,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":53,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":54,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":55,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":56,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":57,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":58,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":59,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":60,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":61,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":62,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":63,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":64,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":65,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":66,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":67,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":68,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":69,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":70,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":71,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":72,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":73,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":74,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":75,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":76,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":77,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":78,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":79,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":80,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":81,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":82,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":83,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":84,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":85,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":86,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":87,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":88,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":89,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":90,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":91,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":92,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":93,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":94,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":95,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":96,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":97,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":98,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":99,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":228,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":229,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":230,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":231,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":232,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":233,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":234,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":235,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":236,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":237,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":238,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":239,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":240,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":241,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":242,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":243,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":244,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":245,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":246,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":247,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":248,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":249,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":250,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":251,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":252,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":253,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":254,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":255,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":256,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":257,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":258,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":259,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":260,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":261,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":262,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":263,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":264,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":265,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":266,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":267,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":268,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":269,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":270,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":271,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":272,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":273,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":274,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":275,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":276,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":277,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":278,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":279,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":280,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":281,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":282,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":283,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":284,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":285,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":286,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":287,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":288,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":289,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":290,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":291,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":292,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":293,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":294,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":295,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":296,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":297,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":298,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":299,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":300,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":301,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":302,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":303,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":304,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":305,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":306,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":307,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":308,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":309,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":310,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":311,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":312,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":313,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":314,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":315,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":316,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":317,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":318,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":319,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":320,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":321,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":322,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":323,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":324,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":325,"url":"https:\/\/docs.atlan.com\/secure-agent","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":326,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":327,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":328,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts","title":"Contracts | Atlan Documentation","text":"Get started Follow these steps to implement contracts in Atlan: Create data contracts Add contract impact analysis in GitHub : Detailed instructions on adding contracts for impact analysis in GitHub."}
{"id":329,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack."}
{"id":330,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts."}
{"id":331,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks."}
{"id":332,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing."}
{"id":333,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management."}
{"id":334,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows."}
{"id":335,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization."}
{"id":336,"url":"https:\/\/docs.atlan.com\/product\/integrations","title":"Integrations | Atlan Documentation","text":"Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack."}
{"id":337,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":338,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":339,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":340,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":341,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":342,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":343,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":344,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":345,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":346,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":347,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":348,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":349,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":350,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":351,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":352,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":353,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":354,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":355,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":356,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":357,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":358,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":359,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":360,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":361,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":362,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":363,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":364,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":365,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":366,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":367,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":368,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":369,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":370,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":371,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":372,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":373,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":374,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":375,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":376,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":377,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":378,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":379,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":380,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":381,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":382,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":383,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":384,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":385,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":386,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":387,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":388,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":389,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":390,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":391,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":392,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":393,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":394,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":395,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":396,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":397,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":398,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":399,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":400,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":401,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":402,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":403,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":404,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":405,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":406,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":407,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":408,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":409,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":410,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":411,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":412,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":413,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":414,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":415,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":416,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":417,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":418,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":419,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":420,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":421,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":422,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":423,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":424,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":425,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":426,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":427,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":428,"url":"https:\/\/developer.atlan.com\/getting-started\/","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":429,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Throughout the portal you can focus on your preferred integration approach (and switch between them as you like): CLI Use the Atlan CLI to manage data contracts for assets in Atlan. Get started with CLI Use the Atlan CLI to manage data contracts for assets in Atlan."}
{"id":430,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with CLI dbt Use dbt's meta field to enrich metadata resources straight from dbt into Atlan. Get started with dbt Use dbt's meta field to enrich metadata resources straight from dbt into Atlan."}
{"id":431,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with dbt Java Pull our Java SDK from Maven Central, just like any other dependency. Get started with Java Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":432,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Java Python Pull our Python SDK from PyPI, just like any other dependency. Get started with Python Pull our Python SDK from PyPI, just like any other dependency."}
{"id":433,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Python Kotlin Pull our Java SDK from Maven Central, just like any other dependency. Get started with Kotlin Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":434,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Kotlin Scala Pull our Java SDK from Maven Central, just like any other dependency. Get started with Scala Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":435,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Scala Clojure Pull our Java SDK from Maven Central, just like any other dependency. Get started with Clojure Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":436,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Clojure Go Pull our Go SDK from GitHub, just like any other dependency. Get started with Go Pull our Go SDK from GitHub, just like any other dependency."}
{"id":437,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with Go Events Tap into events Atlan produces to take immediate action, as metadata changes. Get started with events Tap into events Atlan produces to take immediate action, as metadata changes."}
{"id":438,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with events Raw REST API You can call directly into our REST API, though we would recommend the SDKs. Get started with raw REST APIs Raw REST API You can call directly into our REST API, though we would recommend the SDKs."}
{"id":439,"url":"https:\/\/developer.atlan.com\/sdks\/","title":"Integration options - Developer","text":"Get started with raw REST APIs"}
{"id":440,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":441,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":442,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":443,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":444,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":445,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":446,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":447,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":448,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":449,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":450,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":451,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":452,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Overall site map If you are new to Atlan, or to developing with Atlan, start with one of the following two options. These will set you up to develop with Atlan, step-by-step."}
{"id":453,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Atlan University Self-paced, video-based walkthrough of the essentials of Atlan as a platform. Atlan Platform Essentials certification Self-paced, video-based walkthrough of the essentials of Atlan as a platform."}
{"id":454,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Atlan Platform Essentials certification Introductory walkthrough Documentation-based walkthrough, including step-by-step examples. Start the walkthrough Documentation-based walkthrough, including step-by-step examples."}
{"id":455,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Start the walkthrough Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":456,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":457,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":458,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":459,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":460,"url":"https:\/\/developer.atlan.com\/concepts\/","title":"Developing with Atlan - Developer","text":"Learn more about events"}
{"id":461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#__docusaurus_skipToContent_fallback","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-user-and-role-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-role","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Once you have configured the Snowflake user permissions , you can establish a connection between Atlan and Snowflake. (If you are also using AWS PrivateLink or Azure Private Link for Snowflake, you will need to set that up first, too.) To crawl metadata from Snowflake, review the order of operations and then complete the following steps."}
{"id":612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Select the source To select Snowflake as your source: In the top right of any screen, navigate to New and then click New Workflow. From the list of packages, select Snowflake Assets and click on Setup Workflow."}
{"id":613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Provide credentials Choose your extraction method: In Direct extraction, Atlan connects to your database and crawls metadata directly. In Offline extraction, you will need to first extract metadata yourself and make it available in S3."}
{"id":614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"This is currently only supported when using the information schema extraction method to fetch metadata with basic authentication. In Agent extraction, Atlan's secure agent executes metadata extraction within the organization's environment."}
{"id":615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Direct extraction method To enter your Snowflake credentials: For Account Identifiers (Host) , enter the hostname, AWS PrivateLink endpoint , or Azure Private Link endpoint for your Snowflake instance. For Authentication , choose the method you configured when setting up the Snowflake user : For Basic authentication, enter the Username and Password you configured in either Snowflake or the identity provider."}
{"id":616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"info ð ª Did you know? Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Keypair authentication, enter the Username , Encrypted Private Key , and Private Key Password you configured. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"An empty passphrase will result in workflow failures. To generate an encrypted private key, refer to Snowflake documentation."}
{"id":620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Okta SSO authentication, enter the Username , Password , and Authenticator you configured. The Authenticator will be the Okta URL endpoint of your Okta account , typically in the form of https:\/\/ .okta.com."}
{"id":621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Basic authentication, enter the Username and Password you configured in either Snowflake or the identity provider. info ð ª Did you know?"}
{"id":622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Snowflake recommends transitioning away from basic authentication using username and password. Change to key-pair authentication for enhanced security."}
{"id":623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method. For Basic authentication, enter the Username and Password you configured in either Snowflake or the identity provider."}
{"id":624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"ð ª Did you know? Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Keypair authentication, enter the Username , Encrypted Private Key , and Private Key Password you configured. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"An empty passphrase will result in workflow failures. To generate an encrypted private key, refer to Snowflake documentation."}
{"id":628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Keypair authentication, enter the Username , Encrypted Private Key , and Private Key Password you configured. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"An empty passphrase will result in workflow failures. To generate an encrypted private key, refer to Snowflake documentation."}
{"id":630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Okta SSO authentication, enter the Username , Password , and Authenticator you configured. The Authenticator will be the Okta URL endpoint of your Okta account , typically in the form of https:\/\/ .okta.com."}
{"id":631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Okta SSO authentication, enter the Username , Password , and Authenticator you configured. The Authenticator will be the Okta URL endpoint of your Okta account , typically in the form of https:\/\/ .okta.com."}
{"id":632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Role , select the Snowflake role through which the crawler should run. For Warehouse , select the Snowflake warehouse in which the crawler should run."}
{"id":633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Click Test Authentication to confirm connectivity to Snowflake using these details. Once successful, at the bottom of the screen, click Next."}
{"id":634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Offline extraction method Atlan supports the offline extraction method for fetching metadata from Snowflake. This method uses Atlan's metadata-extractor tool to fetch metadata."}
{"id":635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You will need to first extract the metadata yourself and then make it available in S3. To enter your S3 details: For Bucket name , enter the name of your S3 bucket."}
{"id":636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If you are reusing Atlan's S3 bucket, you can leave this blank. For Bucket prefix , enter the S3 prefix under which all the metadata files exist."}
{"id":637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"These include databases.json , columns- .json , and so on. For Bucket region , enter the name of the S3 region."}
{"id":638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"When complete, at the bottom of the screen, click Next. Configure the connection To complete the Snowflake connection configuration: Provide a Connection Name that represents your source environment."}
{"id":639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For example, you might use values like production , development , gold , or analytics. Provide a Connection Name that represents your source environment."}
{"id":640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For example, you might use values like production , development , gold , or analytics. (Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins."}
{"id":641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"danger If you do not specify any user or group, nobody will be able to manage the connection - not even admins. (Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins."}
{"id":642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If you do not specify any user or group, nobody will be able to manage the connection - not even admins. (Optional) To prevent users from querying any Snowflake data, change Allow SQL Query to No."}
{"id":643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(Optional) To prevent users from querying any Snowflake data, change Allow SQL Query to No. (Optional) To prevent users from previewing any Snowflake data, change Allow Data Preview to No."}
{"id":644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(Optional) To prevent users from previewing any Snowflake data, change Allow Data Preview to No. At the bottom of the screen, click Next to proceed."}
{"id":645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"At the bottom of the screen, click Next to proceed. Agent extraction method Atlan supports using a Secure Agent for fetching metadata from Snowflake."}
{"id":646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"To use a Secure Agent, follow these steps: Select the Agent tab. Configure the Snowflake data source by adding the secret keys for your secret store."}
{"id":647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For details on the required fields, refer to the Direct extraction section. Complete the Secure Agent configuration by following the instructions in the How to configure Secure Agent for workflow execution guide."}
{"id":648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Click Next after completing the configuration. Configure the crawler When modifying an existing Snowflake connection, switching to a different extraction method will delete and recreate all assets in the existing connection."}
{"id":649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If you'd like to change the extraction method, contact Atlan support for assistance. Before running the Snowflake crawler, you can further configure it."}
{"id":650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You must select the Extraction method you configured when you set up Snowflake : For Information Schema method , keep the default selection. Change to Account Usage method and specify the following: Database Name of the copied Snowflake database Schema Name of the copied ACCOUNT_USAGE schema Incremental extraction Public preview - Toggle incremental extraction for faster and more efficient metadata extraction."}
{"id":651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Database Name of the copied Snowflake database Schema Name of the copied ACCOUNT_USAGE schema Incremental extraction Public preview - Toggle incremental extraction for faster and more efficient metadata extraction. You can override the defaults for any of the remaining options: For Asset selection , select a filtering option: To select the assets you want to include in crawling, click Include by hierarchy and filter for assets down to the database or schema level."}
{"id":652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(This will default to all assets, if none are specified.) To have the crawler include Databases , Schemas , or Tables & Views based on a naming convention, click Include by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_DB.* for Databases will include all the matching databases and their child assets. To select the assets you want to exclude from crawling, click Exclude by hierarchy and filter for assets down to the database or schema level."}
{"id":653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(This will default to no assets, if none are specified.) To have the crawler ignore Databases , Schemas , or Tables & Views based on a naming convention, click Exclude by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_TABLES.* for Tables & Views will exclude all the matching tables and views. Click + to add more filters."}
{"id":654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If you add multiple filters, assets will be crawled based on matching all the filtering conditions you have set. For Asset selection , select a filtering option: To select the assets you want to include in crawling, click Include by hierarchy and filter for assets down to the database or schema level."}
{"id":655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(This will default to all assets, if none are specified.) To have the crawler include Databases , Schemas , or Tables & Views based on a naming convention, click Include by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_DB.* for Databases will include all the matching databases and their child assets. To select the assets you want to exclude from crawling, click Exclude by hierarchy and filter for assets down to the database or schema level."}
{"id":656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"(This will default to no assets, if none are specified.) To have the crawler ignore Databases , Schemas , or Tables & Views based on a naming convention, click Exclude by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_TABLES.* for Tables & Views will exclude all the matching tables and views. Click + to add more filters."}
{"id":657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If you add multiple filters, assets will be crawled based on matching all the filtering conditions you have set. To exclude lineage for views in Snowflake, change View Definition Lineage to No."}
{"id":658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"To exclude lineage for views in Snowflake, change View Definition Lineage to No. To import tags from Snowflake to Atlan , change Import Tags to Yes."}
{"id":659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Note the following: If using the Account Usage extraction method, grant the same permissions as required for crawling Snowflake assets to import tags and push updated tags to Snowflake. If using the Information Schema extraction method, note that Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You will need to grant permissions on the account usage schema instead to import tags from Snowflake. danger Object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If your organization does not have Enterprise Edition or higher and you try to import Snowflake tags to Atlan, the Snowflake connection will fail with an error - unable to retrieve tags. To import tags from Snowflake to Atlan , change Import Tags to Yes."}
{"id":662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Note the following: If using the Account Usage extraction method, grant the same permissions as required for crawling Snowflake assets to import tags and push updated tags to Snowflake. If using the Information Schema extraction method, note that Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You will need to grant permissions on the account usage schema instead to import tags from Snowflake. Object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If your organization does not have Enterprise Edition or higher and you try to import Snowflake tags to Atlan, the Snowflake connection will fail with an error - unable to retrieve tags. For Control Config , keep Default for the default configuration or click Custom to further configure the crawler: If you have received a custom crawler configuration from Atlan support, for Custom Config , enter the value provided."}
{"id":665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You can also: Enter {\"ignore-all-case\": true} to enable crawling assets with case-sensitive identifiers. For Enable Source Level Filtering , click True to enable schema-level filtering at source or keep False to disable it."}
{"id":666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Use JDBC Internal Methods , click True to enable JDBC internal methods for data extraction or click False to disable it. For Exclude tables with empty data , change to Yes to exclude any tables and corresponding columns without any data."}
{"id":667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Exclude views , change to Yes to exclude all views from crawling. For Control Config , keep Default for the default configuration or click Custom to further configure the crawler: If you have received a custom crawler configuration from Atlan support, for Custom Config , enter the value provided."}
{"id":668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You can also: Enter {\"ignore-all-case\": true} to enable crawling assets with case-sensitive identifiers. Enter {\"ignore-all-case\": true} to enable crawling assets with case-sensitive identifiers."}
{"id":669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Enable Source Level Filtering , click True to enable schema-level filtering at source or keep False to disable it. For Use JDBC Internal Methods , click True to enable JDBC internal methods for data extraction or click False to disable it."}
{"id":670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"For Exclude tables with empty data , change to Yes to exclude any tables and corresponding columns without any data. For Exclude views , change to Yes to exclude all views from crawling."}
{"id":671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"If an asset appears in both the include and exclude filters, the exclude filter takes precedence. Run the crawler To run the Snowflake crawler, after completing the steps above: To check for any permissions or other configuration issues before running the crawler, click Preflight checks."}
{"id":672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"You can either: To run the crawler once immediately, at the bottom of the screen, click the Run button. To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button."}
{"id":673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"To run the crawler once immediately, at the bottom of the screen, click the Run button. To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button."}
{"id":674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Once the crawler has completed running, you will see the assets in Atlan's asset page! ð Note that the Atlan crawler will currently skip any unsupported data types to ensure a successful workflow run."}
{"id":675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/crawl-snowflake","title":"Crawl Snowflake | Atlan Documentation","text":"Select the source Configure the connection Configure the crawler Run the crawler"}
{"id":676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#create-a-user","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-public-key-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#with-a-password-in-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":826,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Once you've scheduled or run a workflow you can modify its configuration at any time. The configuration that can be modified may vary by workflow but the general steps remain consistent."}
{"id":827,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Modify connectivity To modify the configuration of an existing workflow, complete the following steps. On the left of any screen, navigate to Workflow."}
{"id":828,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Under Monitor select an existing workflow tile. (You may need to expand the run history or filter first.) From the Workflow Run History table, click on the previous run of the workflow you want to modify."}
{"id":829,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"In the upper left of the screen, change to the Config tab. Modify the parts of the workflow configuration you require: Under Credential , use the Edit Credentials button to change the credentials for the source."}
{"id":830,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"danger If you're updating the connection credentials, you may also need to update the metadata filters before running the updated workflow. Atlan currently does not detect changes to your connection settings and update the metadata filters automatically."}
{"id":831,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Under Connection settings , use the Edit button to change the connection details: Modify whether or not querying or data previews are allowed for the source. Modify the query row limit to enable exporting large query results via email."}
{"id":832,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Modify the query timeout limit - expandable up to 60 minutes. Under Connection Admins , click the pencil icon to add or remove connection admins."}
{"id":833,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"danger If you do not specify any user or group, nobody will be able to manage the connection - not even admins. Under Metadata , use the selectors to modify which metadata to include and exclude."}
{"id":834,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"To check for any permissions or other configuration issues before running the workflow, click Preflight checks. Under Credential , use the Edit Credentials button to change the credentials for the source."}
{"id":835,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"danger If you're updating the connection credentials, you may also need to update the metadata filters before running the updated workflow. Atlan currently does not detect changes to your connection settings and update the metadata filters automatically."}
{"id":836,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Under Credential , use the Edit Credentials button to change the credentials for the source. If you're updating the connection credentials, you may also need to update the metadata filters before running the updated workflow."}
{"id":837,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Atlan currently does not detect changes to your connection settings and update the metadata filters automatically. Under Connection settings , use the Edit button to change the connection details: Modify whether or not querying or data previews are allowed for the source."}
{"id":838,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Modify the query row limit to enable exporting large query results via email. Modify the query timeout limit - expandable up to 60 minutes."}
{"id":839,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Under Connection settings , use the Edit button to change the connection details: Modify whether or not querying or data previews are allowed for the source. Modify the query row limit to enable exporting large query results via email."}
{"id":840,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Modify the query timeout limit - expandable up to 60 minutes. Under Connection Admins , click the pencil icon to add or remove connection admins."}
{"id":841,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"danger If you do not specify any user or group, nobody will be able to manage the connection - not even admins. Under Connection Admins , click the pencil icon to add or remove connection admins."}
{"id":842,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"If you do not specify any user or group, nobody will be able to manage the connection - not even admins. Under Metadata , use the selectors to modify which metadata to include and exclude."}
{"id":843,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Under Metadata , use the selectors to modify which metadata to include and exclude. To check for any permissions or other configuration issues before running the workflow, click Preflight checks."}
{"id":844,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"To check for any permissions or other configuration issues before running the workflow, click Preflight checks. Once you've made your updates, click the Update button to save the changes."}
{"id":845,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"You can optionally run the workflow with the new configuration immediately. You will need to confirm your changes by clicking the Yes button."}
{"id":846,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"Note that some workflow changes may take a few minutes to come into effect. You can optionally run the workflow with the new configuration immediately."}
{"id":847,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"You will need to confirm your changes by clicking the Yes button. Note that some workflow changes may take a few minutes to come into effect."}
{"id":848,"url":"https:\/\/docs.atlan.com\/product\/connections\/how-tos\/manage-connectivity","title":"Manage connectivity | Atlan Documentation","text":"That's it - next time you run the workflow, or it runs on its schedule, it will use your changes! ð If you modify the Metadata portion, any previously crawled metadata that is now excluded will be archived on the next workflow run."}
{"id":849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#managed-through-your-identity-provider-idp-","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-role-to-user","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#configure-oauth-client-credentials-flow-with-microsoft-entra-id","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#choose-metadata-fetching-method","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-account-usage-method","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Missing warehouse grants The user doesn t have USAGE and OPERATE grants on a warehouse. Grant warehouse access to the role : GRANT OPERATE , USAGE ON WAREHOUSE \" \" TO ROLE atlan_user_role ; Grant warehouse access to the role : Then, ensure that you grant the role to the new user : GRANT ROLE atlan_user_role TO USER atlan_user ; Then, ensure that you grant the role to the new user : Missing authorized access to SNOWFLAKE.ACCOUNT_USAGE schema The user doesn t have authorized access to the SNOWFLAKE.ACCOUNT_USAGE database Reach out to your account admin to grant imported privileges on the Snowflake database to the role: USE ROLE ACCOUNTADMIN ; GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE atlan_user_role ; Reach out to your account admin to grant imported privileges on the Snowflake database to the role: If using a copied database , you'll need to grant the following permissions: GRANT USAGE ON DATABASE \" \" TO ROLE atlan_user_role ; GRANT USAGE ON SCHEMA \" \" IN DATABASE \" \" TO ROLE atlan_user_role ; GRANT REFERENCES ON ALL VIEWS IN DATABASE \" \" TO ROLE atlan_user_role ; If using a copied database , you'll need to grant the following permissions: Missing usage grants on databases and\/or schemas The user doesn't have usage grants to the databases ` $missingDatabases ` and schemas ` $missingSchemas Grant missing permissions listed here for information schema extraction method."}
{"id":1100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Atlan IP not allowlisted Atlan's current location or network isn't recognized by Snowflake's security settings. This can happen if Atlan's IP address isn't on the list of allowed addresses in Snowflake's network policies."}
{"id":1101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist. Contact Atlan support to obtain Atlan's IP addresses."}
{"id":1102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"The username or the password provided to connect to the Snowflake account is incorrect. Sign into the Snowflake account for the specified host and verify that the username and password are correct."}
{"id":1103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"You can also create a new user, if required, by following the steps here. Missing or unauthorized role The role specified in your connection configuration doesn't exist in Snowflake or your user account doesn't have grant to use this role."}
{"id":1104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"If the role does not exist or is missing the required grants, create a role and then grant the role to the user. User account locked The user account you're using to connect to Snowflake has been locked temporarily because of multiple incorrect login attempts."}
{"id":1105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Wait for the user account to unlock or create a different user account to continue. Missing or unauthorized warehouse The warehouse specified in your connection configuration doesn't exist in Snowflake or your user account doesn't have grant to use this warehouse."}
{"id":1106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Ensure that the warehouse name is configured correctly. Ensure that the warehouse name is configured correctly."}
{"id":1107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Update the warehouse name in the configuration if your account is using a different warehouse. Create a role and then grant the role to the user for the updated warehouse."}
{"id":1108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Update the warehouse name in the configuration if your account is using a different warehouse. Create a role and then grant the role to the user for the updated warehouse."}
{"id":1109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Missing access to non-system databases or schemas The configured user doesn't have usage grants to any database or schema. or The configured user doesn't have usage grants to any non-system database or schema."}
{"id":1110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"This pertains to the information schema method of fetching metadata. Ensure that the user has authorized access to the databases and schemas to be crawled."}
{"id":1111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Grant the requisite permissions as outlined here. Check the grants on the role attached to the user defined for the crawler."}
{"id":1112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Ensure the missing database or schema is present in these grants. SHOW GRANTS TO ROLE atlan_user_role ; Check the grants on the role attached to the user defined for the crawler."}
{"id":1113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Ensure the missing database or schema is present in these grants. When using incremental extraction, consider running a one-time full extraction to capture any newly introduced metadata."}
{"id":1114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"When using incremental extraction, consider running a one-time full extraction to capture any newly introduced metadata. Make sure the role attached to the user defined for the crawler has grants for future tables and views being created in the database: GRANT USAGE ON FUTURE SCHEMAS IN DATABASE \" \" TO ROLE atlan_user_role ; GRANT REFERENCES ON FUTURE TABLES IN DATABASE \" \" TO ROLE atlan_user_role ; GRANT REFERENCES ON FUTURE VIEWS IN DATABASE \" \" TO ROLE atlan_user_role ; GRANT REFERENCES ON FUTURE EXTERNAL TABLES IN DATABASE \" \" TO ROLE atlan_user_role ; Make sure the role attached to the user defined for the crawler has grants for future tables and views being created in the database: Make sure you run the below commands as well so that new tables and views you've created in-between are also visible to the user: GRANT USAGE ON ALL SCHEMAS IN DATABASE \" \" TO role atlan_user_role ; GRANT REFERENCES ON ALL TABLES IN DATABASE \" \" TO role atlan_user_role ; GRANT REFERENCES ON ALL EXTERNAL TABLES IN DATABASE \" \" TO atlan_user_role ; GRANT REFERENCES ON ALL VIEWS IN DATABASE \" \" TO role atlan_user_role ; Make sure you run the below commands as well so that new tables and views you've created in-between are also visible to the user: The query miner only mines query history for up to the previous two weeks."}
{"id":1115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"The miner will not mine any queries that ran before that time window. If the queries that created your assets ran before that time window, lineage for those assets will not be present."}
{"id":1116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"To mine more than the previous two weeks of query history, either use S3-based query mining or contact Atlan support. Note that Snowflake itself only retains query history for so long as well, though."}
{"id":1117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Once Snowflake itself no longer contains the query history we will be unable to mine it for lineage. Lineage is unsupported for parameterized queries."}
{"id":1118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"Snowflake currently does not resolve values for parameterized queries before logging them in query history. This limits Atlan from generating lineage in such cases."}
{"id":1119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"When using the account usage extraction method, there are currently some limitations. We are working with Snowflake to find workarounds for crawling the following: External table location data Procedures Primary key designation External table location data Primary key designation Furthermore, only database-level filtering is currently possible."}
{"id":1120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"When using the account usage method for fetching metadata, Atlan requires access to the following views in Snowflake: For the crawler: DATABASES , SCHEMATA , TABLES , VIEWS , COLUMNS , and PIPES For the miner and popularity metrics : QUERY_HISTORY , ACCESS_HISTORY , and SESSIONS This error can occur when you're connecting to Snowflake through Okta SSO and enter the URL of your Snowflake instance in a format different from the one used in Okta. Snowflake follows two URL formats: Legacy format -."}
{"id":1121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":".snowflakecomputing.com or. ."}
{"id":1122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":".snowflakecomputing.com New URL format - - .snowflakecomputing.com Ensure that you're using the same Snowflake URL format in Snowflake and Okta. Refer to Snowflake documentation to learn more."}
{"id":1123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"If you're getting the following error messages - java.net.UnknownHostException and Name or service not known - this is a known error for users who have upgraded to the Snowflake JDBC driver version 3.13.25., have underscores in their account name, and connect to their Snowflake accounts over private link (for example, https:\/\/my_account.us-west-2.privatelink.snowflakecomputing.com ). If your Snowflake account name has an underscore - for example, my_account - the updated JDBC driver will automatically convert underscores to dashes or hyphens -."}
{"id":1124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"This does not affect normal URLs because Snowflake accepts URLs with both hyphens and underscores. For private link users, however, the JDBC driver will return an error if there are underscores present in the account name and the connection will fail."}
{"id":1125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/troubleshooting\/troubleshooting-snowflake-connectivity","title":"Troubleshooting Snowflake connectivity | Atlan Documentation","text":"To troubleshoot further, refer to Snowflake documentation ."}
{"id":1126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-assets-generate-lineage-and-import-tags","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Once you have crawled assets from Snowflake , you can mine its query history to construct lineage. To mine lineage from Snowflake, review the order of operations and then complete the following steps."}
{"id":1177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Select the miner To select the Snowflake miner: In the top right of any screen, navigate to New and then click New Workflow. From the filters along the top, click Miner."}
{"id":1178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"From the list of packages, select Snowflake Miner and then click Setup Workflow. Configure the miner To configure the Snowflake miner: For Connection , select the connection to mine."}
{"id":1179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"(To select a connection, the crawler must have already run.) For Connection , select the connection to mine. (To select a connection, the crawler must have already run.) For Miner Extraction Method , select Source , Agent , or see the separate instructions for the S3 miner."}
{"id":1180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"For Miner Extraction Method , select Source , Agent , or see the separate instructions for the S3 miner. For Snowflake Database : If the connection is configured with access to the snowflake database , choose Default."}
{"id":1181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"If the connection can only access a separate cloned database , choose Cloned Database. For Snowflake Database : If the connection is configured with access to the snowflake database , choose Default."}
{"id":1182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"If the connection can only access a separate cloned database , choose Cloned Database. If you are using a cloned database, enter the name of the cloned database in Database Name and the name of the cloned schema in Schema Name."}
{"id":1183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"If you are using a cloned database, enter the name of the cloned database in Database Name and the name of the cloned schema in Schema Name. For Start time , choose the earliest date from which to mine query history."}
{"id":1184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"info ð ª Did you know? The miner restricts you to only querying the past two weeks of query history."}
{"id":1185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"If you need to query more history, for example in an initial load, consider using the S3 miner first. After the initial load, you can modify the miner's configuration to use query history extraction."}
{"id":1186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"For Start time , choose the earliest date from which to mine query history. ð ª Did you know?"}
{"id":1187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"The miner restricts you to only querying the past two weeks of query history. If you need to query more history, for example in an initial load, consider using the S3 miner first."}
{"id":1188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"After the initial load, you can modify the miner's configuration to use query history extraction. To check for any permissions or other configuration issues before running the miner, click Preflight checks."}
{"id":1189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"To check for any permissions or other configuration issues before running the miner, click Preflight checks. At the bottom of the screen, click Next to proceed."}
{"id":1190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"At the bottom of the screen, click Next to proceed. Agent extraction method Atlan supports using a Secure Agent for mining query history from Snowflake."}
{"id":1191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"To use a Secure Agent, follow these steps: Query redaction is a compute-intensive operation and can impact workflow performance. To use this feature effectively, assign at least 4 CPU cores to the agent."}
{"id":1192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Select the Agent tab. Configure Query Redaction : By default, Query Redaction is turned off."}
{"id":1193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"You may choose to configure query redaction: Set Enable Query Redaction flag to true if you want to redact PII data from query history before ingesting it to Atlan. Configure PII Patterns for Query Redaction : Define patterns to identify and redact sensitive information using the following JSON format: { \"pii_patterns\" : [ { \"name\" : \"email\" , \"regex\" : \"\\\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\\\.[A-Za-z]{2,}\\\\b\" , \"replacement\" : \"[EMAIL]\" } ] } Set Enable Query Redaction flag to true if you want to redact PII data from query history before ingesting it to Atlan."}
{"id":1194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Configure PII Patterns for Query Redaction : Define patterns to identify and redact sensitive information using the following JSON format: { \"pii_patterns\" : [ { \"name\" : \"email\" , \"regex\" : \"\\\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\\\.[A-Za-z]{2,}\\\\b\" , \"replacement\" : \"[EMAIL]\" } ] } Configure the Snowflake data source by adding the secret keys for your secret store. For details on the required fields, refer to the connection configuration used when crawling Snowflake."}
{"id":1195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Complete the Secure Agent configuration by following the instructions in the How to configure Secure Agent for workflow execution guide. Click Next after completing the configuration."}
{"id":1196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"If running the miner for the first time, Atlan recommends setting a start date around three days prior to the current date and then scheduling it daily to build up to two weeks of query history. Mining two weeks of query history on the first miner run may cause delays."}
{"id":1197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"For all subsequent runs, Atlan requires a minimum lag of 24 to 48 hours to capture all the relevant transformations that were part of a session. Learn more about the miner logic here."}
{"id":1198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Configure the miner behavior To configure the Snowflake miner behavior: (Optional) For Calculate popularity , keep True to retrieve usage and popularity metrics for your Snowflake assets from query history. For Excluded Users , type the names of users to be excluded while calculating usage metrics for Snowflake assets."}
{"id":1199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Press Enter after each name to add more names. For Excluded Users , type the names of users to be excluded while calculating usage metrics for Snowflake assets."}
{"id":1200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"Press Enter after each name to add more names. (Optional) For Advanced Config , keep Default for the default configuration or click Custom to configure the miner: If Atlan support has provided you with a custom control configuration, enter the configuration into the Custom Config box."}
{"id":1201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"You can also enter { ignore-all-case : true} to enable crawling assets with case-sensitive identifiers. For Popularity Window (days) , 90 days is the maximum limit."}
{"id":1202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"You can set a shorter popularity window of less than 90 days. If Atlan support has provided you with a custom control configuration, enter the configuration into the Custom Config box."}
{"id":1203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"You can also enter { ignore-all-case : true} to enable crawling assets with case-sensitive identifiers. For Popularity Window (days) , 90 days is the maximum limit."}
{"id":1204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"You can set a shorter popularity window of less than 90 days. Run the miner To run the Snowflake miner, after completing the configuration steps: To run the miner once immediately, at the bottom of the screen, click the Run button."}
{"id":1205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"To schedule the miner to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule & Run button. Once the miner has completed running, you can see lineage for Snowflake assets that were created in Snowflake between the start time and when the miner ran!"}
{"id":1206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/mine-snowflake","title":"Mine Snowflake | Atlan Documentation","text":"ð Select the miner Configure the miner Configure the miner behavior Run the miner"}
{"id":1207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. Atlan enables you to import your Snowflake tags , update your Snowflake assets with the imported tags, and push the tag updates back to Snowflake: Import tags - crawl Snowflake tags from Snowflake to Atlan Reverse sync - sync Snowflake tag updates from Atlan to Snowflake Once you've imported your Snowflake tags to Atlan: Your Snowflake assets in Atlan are automatically enriched with their Snowflake tags."}
{"id":1208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Imported Snowflake tags are mapped to corresponding Atlan tags through case-insensitive name match - multiple Snowflake tags can be matched to a single tag in Atlan. You can also attach Snowflake tags , including tag values, to your Snowflake assets in Atlan - allowing you to categorize your assets at a more granular level."}
{"id":1209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Atlan supports: Allowed values : attach an allowed value from a predefined list of values imported from Snowflake. Tag values: enter any value in Atlan while attaching or editing imported Snowflake tags on an asset."}
{"id":1210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Allowed values : attach an allowed value from a predefined list of values imported from Snowflake. Tag values: enter any value in Atlan while attaching or editing imported Snowflake tags on an asset."}
{"id":1211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"You can enable reverse sync to push any tag updates for your Snowflake assets back to Snowflake - including allowed and tag values added to assets in Atlan. You can filter your assets by Snowflake tags and tag and allowed values."}
{"id":1212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Enabling reverse sync only updates existing tags in Snowflake. It neither creates nor deletes any tags in Snowflake."}
{"id":1213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Additional privileges are only required when using the information schema method for fetching metadata. This is because Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , any permissions required are already set. Account usage method Before you can import tags from Snowflake, you need to do the following: Create tags or have existing tags in Snowflake."}
{"id":1215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Grant the same permissions as required for crawling Snowflake assets to import tags and push updated tags to Snowflake. Information schema method Before you can import tags from Snowflake, you need to do the following: Create tags or have existing tags in Snowflake."}
{"id":1216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Grant additional permissions to import tags from Snowflake. Grant additional permissions to push updated tags to Snowflake."}
{"id":1217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Import Snowflake tags to Atlan You need to be an admin user in Atlan to import Snowflake tags to Atlan. You also need to work with your Snowflake administrator to grant additional permissions to import tags from Snowflake - you may not have access yourself."}
{"id":1218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"You can import your Snowflake tags to Atlan through one-way tag sync. The synced Snowflake tags are matched to corresponding tags in Atlan through case-insensitive name match and your Snowflake assets are enriched with their synced tags from Snowflake."}
{"id":1219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"To import Snowflake tags to Atlan, you can either: Create a new Snowflake workflow and configure the crawler to import tags. Modify the crawler's configuration for an existing Snowflake workflow to change Import Tags to Yes."}
{"id":1220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"If you subsequently modify the workflow to disable tag import, for any tags already imported, Atlan preserves those tags. Once the crawler has completed running, tags imported from Snowflake are available to use for tagging assets!"}
{"id":1221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"ð View Snowflake tags in Atlan Once you've imported your Snowflake tags, you can view and manage your Snowflake tags in Atlan. To view Snowflake tags: From the left menu of any screen, click Governance."}
{"id":1222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Under the Governance heading of the _Governance cente_r, click Tags. (Optional) Under Tags , click the funnel icon to filter tags by source type."}
{"id":1223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Click Snowflake to filter for tags imported from Snowflake. From the left menu under Tags , select a synced tag - synced tags display the Snowflake ï¸ icon next to the tag name."}
{"id":1224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"(Optional) Click the Linked assets tab to view linked assets for your Snowflake tag. (Optional) In the top right, click the pencil icon to add a description and change the tag icon."}
{"id":1225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"You can't rename tags synced from Snowflake. Push tag updates to Snowflake Any admin or member user in Atlan can configure reverse sync for tag updates to Snowflake."}
{"id":1226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"You also need to work with your Snowflake administrator to grant additional permissions to push updates - you may not have access yourself. Reverse sync is currently only available for imported Snowflake tags in Atlan."}
{"id":1227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"The imported tags display a Snowflake ï¸ icon next to the tag name. If using the account usage method , expect a data latency of up to 3 hours for reverse tag sync to be successful."}
{"id":1228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"You can enable reverse sync for your imported Snowflake tags in Atlan and push all tag updates for your Snowflake assets back to source. Once you have enabled reverse sync, any Snowflake assets with tags updated in Atlan are also updated in Snowflake."}
{"id":1229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"To enable reverse sync for imported Snowflake tags: From the left menu of any screen, click Governance. Under the Governance heading of the _Governance cente_r, click Tags."}
{"id":1230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"(Optional) Under Tags , click the funnel icon to filter tags by source type. Click Snowflake to filter for tags imported from Snowflake."}
{"id":1231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"In the left menu under Tags , select a synced Snowflake tag - synced tags display the Snowflake ï¸ icon next to the tag name. Under Synced tags , in the upper right, turn on Enable reverse sync to synchronize tag updates from Atlan to Snowflake."}
{"id":1232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"In the advanced settings, you can also enable concatenation to support multiple tag values for a single column. For detailed information about multiple tag values and concatenation, see Multiple tag values and concatenation."}
{"id":1233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"In the corresponding confirmation dialog, click Yes, enable it to enable reverse tag sync or click Cancel. Now when you attach Snowflake tags to your Snowflake assets in Atlan, these tag updates are also pushed to Snowflake!"}
{"id":1234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"ð Enabling reverse sync won't trigger any updates in Snowflake until synced tags are attached to Snowflake assets in Atlan. For any questions about managing Snowflake tags, head over here."}
{"id":1235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/manage-snowflake-tags","title":"Manage Snowflake tags | Atlan Documentation","text":"Import Snowflake tags to Atlan View Snowflake tags in Atlan Push tag updates to Snowflake"}
{"id":1236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-streams","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#grant-permissions-for-information-schema-method","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-existing-assets","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-crawl-future-assets","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#to-mine-query-history-for-lineage","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-existing-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-preview-and-query-future-assets-1","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-import-snowflake-tags","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-push-updated-tags-to-snowflake","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-dynamic-tables","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-iceberg-tables","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#optional-to-crawl-snowflake-stages","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"You need your Snowflake administrator to run these commands - you may not have access yourself. Create user and role in Snowflake Create a role and user in Snowflake using the following commands: Create role Create a role in Snowflake using the following commands: Replace with the default warehouse to use when running the Snowflake crawler."}
{"id":1937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Atlan requires the following privileges to: OPERATE enables Atlan to start the virtual warehouse to fetch metadata if the warehouse has stopped. USAGE enables Atlan to show or list metadata from Snowflake."}
{"id":1938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"This in turn enables the Snowflake crawler to run the SHOW query. Create a user Create a separate user to integrate into Atlan, using one of the following 3 options: See Snowflake's official guide for details on generating an RSA key-pair."}
{"id":1939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a key-pair, replace the value for rsa_public_key with the public key and run the following: Learn more about the SERVICE type property in Snowflake documentation. Atlan only supports encrypted private keys with a non-empty passphrase - generally recommended as more secure."}
{"id":1940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"An empty passphrase results in workflow failures. To generate an encrypted private key, omit the -nocrypt option."}
{"id":1941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Refer to Snowflake documentation to learn more. Snowflake recommends transitioning away from basic authentication using username and password."}
{"id":1942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Change to key-pair authentication for enhanced security. For any existing Snowflake workflows, you can modify the crawler configuration to update the authentication method."}
{"id":1943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To create a user with a password, replace and run the following: Learn more about the LEGACY_SERVICE type property in Snowflake documentation. This method is currently only available if Okta is your IdP (Snowflake supports) authenticating natively through Okta : Create a user in your identity provider (IdP) and use federated authentication in Snowflake."}
{"id":1944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"The password for this user must be maintained solely in the IdP and multi-factor authentication (MFA) must be disabled. Grant role to user To grant the atlan_user_role to the new user: Configure OAuth (client credentials flow) with Microsoft Entra ID To configure OAuth authentication using Microsoft Entra ID (formerly Azure AD) with the client credentials flow: Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions Follow Snowflake's documentation to: Register a new application in Microsoft Entra ID Collect the client ID , tenant ID , and client secret Add the required API permissions In Snowflake, create a security integration using the following: CREATE SECURITY INTEGRATION external_oauth_azure_ad TYPE = external_oauth ENABLED = true EXTERNAL_OAUTH_TYPE = azure EXTERNAL_OAUTH_ISSUER = '\\ ' EXTERNAL_OAUTH_JWS_KEYS_URL = '\\ ' EXTERNAL_OAUTH_AUDIENCE_LIST = ( '\\ ' ) EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'sub' EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name' ; Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app In Snowflake, create a security integration using the following: Replace the placeholders with actual values from your Azure AD app: Your tenant's OAuth 2.0 issuer URL Azure JWKs URI Application ID URI of the Azure app Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: CREATE USER oauth_svc_user WITH LOGIN_NAME = '\\ ' -- Use Azure client OBJECT ID DEFAULT_ROLE = \\ DEFAULT_WAREHOUSE = \\ ; Create a Snowflake user with a login name that exactly matches the Azure AD client object ID: Grant the configured role to this user: GRANT ROLE \\ TO USER oauth_svc_user ; Grant the configured role to this user: Choose metadata fetching method Atlan supports two methods for fetching metadata from Snowflake - account usage and information schema."}
{"id":1945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"You should choose one of these two methods to set up Snowflake: Grant permissions for account usage method If you want to set warehouse timeouts when using this method, set a large value initially for the workflow to succeed. Once the workflow has succeeded, adjust the value to be twice the extraction time."}
{"id":1946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"This method uses the views in SNOWFLAKE.ACCOUNT_USAGE (or a copied version of this schema) to fetch the metadata from Snowflake into Atlan. You can be more granular with permissions using this method, but there are limitations with this approach."}
{"id":1947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To crawl assets, generate lineage, and import tags If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"If you're using the account usage method to crawl metadata in Atlan or you have configured the Snowflake miner , you need to grant the same permissions to import tags as required for crawling Snowflake assets. Note that object tagging in Snowflake currently requires Enterprise Edition or higher."}
{"id":1949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database. To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles."}
{"id":1951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name. Replace with the copied Snowflake ACCOUNT_USAGE schema name."}
{"id":1952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"The grants for the copied version can't be used on the original SNOWFLAKE database. This is because Snowflake produces an error that granular grants can't be given to imported databases."}
{"id":1953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. To crawl streams To crawl streams, provide the following permissions: To crawl current streams: Replace with the Snowflake database name."}
{"id":1955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the Snowflake database name. To crawl future streams: To crawl future streams: Replace with the Snowflake database name."}
{"id":1956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions."}
{"id":1957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead. Replace with the database you want to be able to preview and query in Atlan."}
{"id":1958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. Grant permissions for information schema method This method uses views in the INFORMATION_SCHEMA schema in Snowflake databases to fetch metadata."}
{"id":1959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"You still need to grant specific permissions to enable Atlan to crawl metadata, preview data, and query data with this method. To crawl existing assets Grant these permissions to crawl assets that already exist in Snowflake."}
{"id":1960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"If you also want to be able to preview and query the data, you can use the preview and query existing assets permissions instead. Grant permissions to crawl existing assets: Replace with the database you want to be available in Atlan."}
{"id":1961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to integrate into Atlan.) Grant permissions to crawl functions: Replace with the database you want to be available in Atlan. (Repeat the statements for every database you wish to integrate into Atlan.) For secure user-defined functions (UDFs), grant OWNERSHIP permissions to retrieve metadata: Replace the placeholders with the appropriate values: : The name of the schema that contains the user-defined function (UDF)."}
{"id":1962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":": The name of the secure UDF that requires ownership permissions. : The role that gets assigned ownership of the secure UDF."}
{"id":1963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"The statements given on this page apply to all schemas, tables, and views in a database in Snowflake. If you want to limit access to only certain objects, you can instead specify the exact objects individually as well."}
{"id":1964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To crawl future assets To crawl assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To grant permissions at a database level: Replace with the database you want to crawl in Atlan. (Repeat the statements for every database you want to integrate into Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan."}
{"id":1967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to integrate into Atlan.) To mine query history for lineage To also mine Snowflake's query history (for lineage), add these permissions. You can use either option: To mine query history direct from Snowflake's internal tables: To mine query history from a cloned or copied set of tables, where you can also remove any sensitive data: Replace with the name of the cloned database, and with the name of the cloned schema containing account usage details."}
{"id":1968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"When using a cloned or copied version, verify that the table or view definition remains unchanged as in your SNOWFLAKE database. If the format is different."}
{"id":1969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"For example, a column is missing and it no longer qualifies as a clone. (Optional) To preview and query existing assets To query and preview data within assets that already exist in Snowflake, add these permissions: Replace with the database you want to be able to preview and query in Atlan."}
{"id":1970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database you wish to preview and query in Atlan.) (Optional) To preview and query future assets To query and preview data within assets that may be created in the future in Snowflake, add these permissions. Again, if you want to also be able to preview and query the data for future assets, you can add the preview and query future assets permissions instead."}
{"id":1971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the database you want to be able to preview and query in Atlan. (Repeat the statements for every database you want to preview and query in Atlan.) For any future grants defined at a schema level to a different role, the schema-level grants take precedence over the database-level grants and any future grants defined at a database level to the Atlan role get ignored."}
{"id":1972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To learn more, refer to Snowflake documentation. To grant permissions at a schema level: Replace with the database and with the schema you want to be able to preview and query in Atlan."}
{"id":1973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Repeat the statements for every database and schema you want to preview and query in Atlan.) Verify that all the assets you'd like to crawl are present in these grants by checking the grants on the user role defined for the crawler. (Optional) To import Snowflake tags Snowflake stores all tag objects in the ACCOUNT_USAGE schema."}
{"id":1974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Note that object tagging in Snowflake currently requires Enterprise Edition or higher. To import tags from Snowflake , grant these permissions: To use the default SNOWFLAKE database and ACCOUNT_USAGE schema and also mine Snowflake's query history (for lineage), grant these permissions: The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. The ACCOUNTADMIN role is required to grant privileges on the SNOWFLAKE database due to the following reasons: By default, only the ACCOUNTADMIN role can access the SNOWFLAKE database."}
{"id":1976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To enable other roles to access the database and schemas and query the views, a user with the ACCOUNTADMIN role needs to grant IMPORTED PRIVILEGES on the SNOWFLAKE database to the desired roles. To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: To use a copied or cloned version of this default schema, where you can also remove any sensitive data for security purposes, grant these permissions: Replace with the copied Snowflake database name."}
{"id":1977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Replace with the copied Snowflake ACCOUNT_USAGE schema name. The grants for the copied version can't be used on the original SNOWFLAKE database."}
{"id":1978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"This is because Snowflake produces an error that granular grants can't be given to imported databases. (Optional) To push updated tags to Snowflake To push tags updated for assets in Atlan to Snowflake , grant these permissions: You can learn more about tag privileges from Snowflake documentation."}
{"id":1979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl dynamic tables Atlan currently supports fetching metadata for dynamic tables using the MONITOR privilege. Refer to Snowflake documentation to learn more."}
{"id":1980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To crawl existing dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: To crawl future dynamic tables from Snowflake: Grant permissions at a database level: Grant permissions at a schema level: Replace with the database and with the schema you want to crawl in Atlan. (Repeat the statements for every database and schema you want to integrate into Atlan.) (Optional) To crawl Iceberg tables Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method."}
{"id":1981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To crawl Iceberg tables from Snowflake, grant the following permissions: To crawl existing Iceberg tables in Snowflake: To crawl future Iceberg tables in Snowflake: To crawl Iceberg catalog metadata for Iceberg tables in Snowflake: You must first grant permissions to crawl existing Iceberg tables for this permission to work on catalogs. You must also grant permissions to all the catalogs you want to crawl in Atlan individually."}
{"id":1982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"(Optional) To crawl Snowflake stages Atlan supports crawling metadata for Snowflake stages using the USAGE and READ privileges. For more information, see the Snowflake documentation for INFORMATION_SCHEMA.STAGES."}
{"id":1983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"To crawl stages from Snowflake: Grant USAGE and READ privileges on all existing stages at the database level: Replace with the name of your Snowflake database Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Replace with the role you've granted Atlan to use for crawling."}
{"id":1984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Grant USAGE and READ privileges on all future stages at the database level: Grant USAGE and READ privileges on all future stages at the database level: Replace with the name of your Snowflake database Replace with the role you've granted Atlan to use for crawling. Allowlist the Atlan IP If you are using the IP allowlist in your Snowflake instance, you must add the Atlan IP to the allowlist."}
{"id":1985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/how-tos\/set-up-snowflake#allowlist-the-atlan-ip","title":"Set up Snowflake | Atlan Documentation","text":"Please raise a support ticket from within Atlan, or submit a request. (If you aren't using the IP allowlist in your Snowflake instance, you can skip this step.) Create user and role in Snowflake Choose metadata fetching method Grant permissions for account usage method Grant permissions for information schema method Allowlist the Atlan IP"}
{"id":1986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":1987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":1988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":1989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":1990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":1991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":1992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":1993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":1994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":1995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":1996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":1997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":1998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":1999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#__docusaurus_skipToContent_fallback","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#personal-access-token-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#aws-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#azure-service-principal-authentication","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-user-access-to-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":2795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":2796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":2797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":2798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":2800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":2801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":2802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":2804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":2806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":2807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":2808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":2809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":2811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":2813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":2814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":2815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":2816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":2817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":2818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":2819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#generate-a-personal-access-token","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":2820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Once you have configured the Databricks access permissions , you can establish a connection between Atlan and your Databricks instance. (If you are also using AWS PrivateLink or Azure Private Link for Databricks, you will need to set that up first, too.) To crawl metadata from your Databricks instance, review the order of operations and then complete the following steps."}
{"id":2821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Select the source To select Databricks as your source: In the top right corner of any screen, navigate to New and then click New Workflow. In the top right corner of any screen, navigate to New and then click New Workflow."}
{"id":2822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"From the list of packages, select Databricks Assets , and click Setup Workflow. From the list of packages, select Databricks Assets , and click Setup Workflow."}
{"id":2823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Provide credentials Choose your extraction method: In Direct extraction, Atlan connects to your database and crawls metadata directly. Next, select an authentication method: In JDBC , you will need a personal access token and HTTP path for authentication."}
{"id":2824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"In AWS Service , you will need a client ID and client secret for AWS service principal authentication. In Azure Service , you will need a tenant ID, client ID, and client secret for Azure service principal authentication."}
{"id":2825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"In JDBC , you will need a personal access token and HTTP path for authentication. In AWS Service , you will need a client ID and client secret for AWS service principal authentication."}
{"id":2826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"In Azure Service , you will need a tenant ID, client ID, and client secret for Azure service principal authentication. In Offline extraction, you will need to first extract metadata yourself and make it available in S3."}
{"id":2827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"In Agent extraction, Atlan's secure agent executes metadata extraction within the organization's environment. Direct extraction method To enter your Databricks credentials: For Host , enter the hostname, AWS PrivateLink endpoint , or Azure Private Link endpoint for your Databricks instance."}
{"id":2828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Port , enter the port number of your Databricks instance. For Personal Access Token , enter the access token you generated when setting up access."}
{"id":2829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For HTTP Path , enter one of the following: A path starting with \/sql\/1.0\/warehouses to use the Databricks SQL warehouse. A path starting with sql\/protocolv1\/o to use the Databricks interactive cluster."}
{"id":2830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"A path starting with \/sql\/1.0\/warehouses to use the Databricks SQL warehouse. A path starting with sql\/protocolv1\/o to use the Databricks interactive cluster."}
{"id":2831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Click Test Authentication to confirm connectivity to Databricks using these details. Once successful, at the bottom of the screen click Next."}
{"id":2832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Make sure your Databricks instance (SQL warehouse or interactive cluster) is up and running, otherwise the Test Authentication step times out. To enter your Databricks credentials: For Host , enter the hostname or AWS PrivateLink endpoint for your Databricks instance."}
{"id":2833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Port , enter the port number of your Databricks instance. For Client ID , enter the client ID for your AWS service principal."}
{"id":2834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Client Secret , enter the client secret for your AWS service principal. Click Test Authentication to confirm connectivity to Databricks using these details."}
{"id":2835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Once successful, at the bottom of the screen click Next. To enter your Databricks credentials: For Host , enter the hostname or Azure Private Link endpoint for your Databricks instance."}
{"id":2836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Port , enter the port number of your Databricks instance. For Client ID , enter the application (client) ID for your Azure service principal."}
{"id":2837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Client Secret , enter the client secret for your Azure service principal. For Tenant ID , enter the directory (tenant) ID for your Azure service principal."}
{"id":2838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Click Test Authentication to confirm connectivity to Databricks using these details. Once successful, at the bottom of the screen click Next."}
{"id":2839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Offline extraction method Atlan supports the offline extraction method for fetching metadata from Databricks. This method uses Atlan's databricks-extractor tool to fetch metadata."}
{"id":2840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"You need to first extract the metadata yourself and then make it available in S3. To enter your S3 details: For Bucket name , enter the name of your S3 bucket."}
{"id":2841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Bucket prefix , enter the S3 prefix under which all the metadata files exist. These include output\/databricks-example\/catalogs\/success\/result-0.json , output\/databricks-example\/schemas\/{{catalog_name}}\/success\/result-0.json , output\/databricks-example\/tables\/{{catalog_name}}\/success\/result-0.json , and similar files."}
{"id":2842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) For Bucket region , enter the name of the S3 region. When complete, at the bottom of the screen, click Next."}
{"id":2843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Agent extraction method Atlan supports using a Secure Agent for fetching metadata from Databricks. To use a Secure Agent, follow these steps: Select the Agent tab."}
{"id":2844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Configure the Databricks data source by adding the secret keys for your secret store. For details on the required fields, refer to the Direct extraction section."}
{"id":2845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Complete the Secure Agent configuration by following the instructions in the How to configure Secure Agent for workflow execution guide. Click Next after completing the configuration."}
{"id":2846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Configure the connection To complete the Databricks connection configuration: Provide a Connection Name that represents your source environment. For example, you might want to use values like production , development , gold , or analytics."}
{"id":2847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Provide a Connection Name that represents your source environment. For example, you might want to use values like production , development , gold , or analytics."}
{"id":2848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins. danger If you don't specify any user or group, nobody can manage the connection - not even admins."}
{"id":2849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins. If you don't specify any user or group, nobody can manage the connection - not even admins."}
{"id":2850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) To prevent users from querying any Databricks data, change Enable SQL Query to No. (Optional) To prevent users from querying any Databricks data, change Enable SQL Query to No."}
{"id":2851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) To prevent users from previewing any Databricks data, change Enable Data Preview to No. (Optional) To prevent users from previewing any Databricks data, change Enable Data Preview to No."}
{"id":2852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(Optional) To prevent users from running large queries, change Max Row Limit or keep the default selection. (Optional) To prevent users from running large queries, change Max Row Limit or keep the default selection."}
{"id":2853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"At the bottom of the screen, click the Next button to proceed. At the bottom of the screen, click the Next button to proceed."}
{"id":2854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Configure the crawler Before running the Databricks crawler, you can further configure it. System tables extraction method The system metadata extraction method is only available for Unity Catalog-enabled workspaces."}
{"id":2855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"It provides access to detailed metadata from system tables and supports all three authentication types. You can extract metadata from your Databricks workspace using this method."}
{"id":2856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Follow these steps: Set up authentication using one of the following: Personal access token AWS service principal Azure service principal Set up authentication using one of the following: Personal access token AWS service principal Azure service principal The default options can work as is. You may choose to override the defaults for any of the remaining options: For Asset selection , select a filtering option: For SQL warehouse , click the dropdown to select the SQL warehouse you want to configure."}
{"id":2857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To select the assets you want to include in crawling, click Include by hierarchy and filter for assets down to the database or schema level. (This defaults to all assets, if none are specified.) To have the crawler include Databases , Schemas , or Tables & Views based on a naming convention, click Include by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_DB.* for Databases includes all the matching databases and their child assets."}
{"id":2858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To select the assets you want to exclude from crawling, click Exclude by hierarchy and filter for assets down to the database or schema level. (This defaults to no assets, if none are specified.) To have the crawler ignore Databases , Schemas , or Tables & Views based on a naming convention, click Exclude by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_TABLES.* for Tables & Views excludes all the matching tables and views."}
{"id":2859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Click + to add more filters. If you add multiple filters, assets are crawled based on matching all the filtering conditions you have set."}
{"id":2860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To import tags from Databricks to Atlan , change Import Tags to Yes. Note that you must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan."}
{"id":2861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Did you know? If an asset appears in both the include and exclude filters, the exclude filter takes precedence."}
{"id":2862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"The default options can work as is. You may choose to override the defaults for any of the remaining options: For Asset selection , select a filtering option: For SQL warehouse , click the dropdown to select the SQL warehouse you want to configure."}
{"id":2863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To select the assets you want to include in crawling, click Include by hierarchy and filter for assets down to the database or schema level. (This defaults to all assets, if none are specified.) To have the crawler include Databases , Schemas , or Tables & Views based on a naming convention, click Include by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_DB.* for Databases includes all the matching databases and their child assets."}
{"id":2864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To select the assets you want to exclude from crawling, click Exclude by hierarchy and filter for assets down to the database or schema level. (This defaults to no assets, if none are specified.) To have the crawler ignore Databases , Schemas , or Tables & Views based on a naming convention, click Exclude by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_TABLES.* for Tables & Views excludes all the matching tables and views."}
{"id":2865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Click + to add more filters. If you add multiple filters, assets are crawled based on matching all the filtering conditions you have set."}
{"id":2866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To import tags from Databricks to Atlan , change Import Tags to Yes. Note that you must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan."}
{"id":2867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Did you know? If an asset appears in both the include and exclude filters, the exclude filter takes precedence."}
{"id":2868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Asset selection , select a filtering option: For SQL warehouse , click the dropdown to select the SQL warehouse you want to configure. To select the assets you want to include in crawling, click Include by hierarchy and filter for assets down to the database or schema level."}
{"id":2869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(This defaults to all assets, if none are specified.) To have the crawler include Databases , Schemas , or Tables & Views based on a naming convention, click Include by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_DB.* for Databases includes all the matching databases and their child assets. To select the assets you want to exclude from crawling, click Exclude by hierarchy and filter for assets down to the database or schema level."}
{"id":2870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(This defaults to no assets, if none are specified.) To have the crawler ignore Databases , Schemas , or Tables & Views based on a naming convention, click Exclude by regex and specify a regular expression - for example, specifying ATLAN_EXAMPLE_TABLES.* for Tables & Views excludes all the matching tables and views. Click + to add more filters."}
{"id":2871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"If you add multiple filters, assets are crawled based on matching all the filtering conditions you have set. To import tags from Databricks to Atlan , change Import Tags to Yes."}
{"id":2872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Note that you must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan. If an asset appears in both the include and exclude filters, the exclude filter takes precedence."}
{"id":2873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Toggle incremental extraction, for a faster and more efficient metadata extraction. JDBC extraction method The JDBC extraction method uses JDBC queries to extract metadata from your Databricks instance."}
{"id":2874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"This was the original extraction method provided by Databricks. This extraction method is only supported for personal access token authentication."}
{"id":2875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"You can override the defaults for any of these options: To select the assets you want to include in crawling, click Include Metadata. (This will default to all assets, if none are specified.) To select the assets you want to exclude from crawling, click Exclude Metadata."}
{"id":2876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(This will default to no assets if none are specified.) To have the crawler ignore tables and views based on a naming convention, specify a regular expression in the Exclude regex for tables & views field. For View Definition Lineage , keep the default Yes to generate upstream lineage for views based on the tables referenced in the views or click No to exclude from crawling."}
{"id":2877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For Advanced Config , keep Default for the default configuration or click Advanced to further configure the crawler: To enable or disable schema-level filtering at source, click Enable Source Level Filtering and select True to enable it or False to disable it. To enable or disable schema-level filtering at source, click Enable Source Level Filtering and select True to enable it or False to disable it."}
{"id":2878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"REST API extraction method The REST API extraction method uses Unity Catalog to extract metadata from your Databricks instance. This extraction method is supported for all three authentication options: personal access token , AWS service principal , and Azure service principal."}
{"id":2879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"This method is only supported by Unity Catalog-enabled workspaces. If you enable an existing workspace, you also need to upgrade your tables and views to Unity Catalog."}
{"id":2880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"While REST APIs are used to extract metadata, JDBC queries are still used for querying purposes. You can override the defaults for any of these options: Change the extraction method under Extraction method to REST API."}
{"id":2881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To select the assets you want to include in crawling, click Include Metadata. (This will default to all assets, if none are specified.) To select the assets you want to exclude from crawling, click Exclude Metadata."}
{"id":2882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"(This will default to no assets if none are specified.) To import tags from Databricks to Atlan , change Import Tags to Yes. Note that you must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan."}
{"id":2883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"For SQL warehouse , click the dropdown to select the SQL warehouse you have configured. For SQL warehouse , click the dropdown to select the SQL warehouse you have configured."}
{"id":2884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"If an asset appears in both the include and exclude filters, the exclude filter takes precedence. Run the crawler Follow these steps to run the Databricks crawler: To check for any permissions or other configuration issues before running the crawler, click Preflight checks."}
{"id":2885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"You can either: To run the crawler once immediately, at the bottom of the screen, click the Run button. To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button."}
{"id":2886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"To run the crawler once immediately, at the bottom of the screen, click the Run button. To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button."}
{"id":2887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/crawl-databricks","title":"Crawl Databricks | Atlan Documentation","text":"Once the crawler has completed running, you will see the assets in Atlan's asset page! ð Select the source Configure the connection Configure the crawler Run the crawler"}
{"id":2888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":2889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":2890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":2892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":2893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":2894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":2895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":2896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":2897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":2898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":2900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":2902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":2903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":2904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":2905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":2906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":2907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":2908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":2909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":2911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":2912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":2913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":2914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":2915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":2917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":2918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":2919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":2920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":2921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":2922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":2923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":2924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":2925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":2926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":2927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":2929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":2930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":2931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":2932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":2933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":2934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":2935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":2936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":2937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":2938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":2939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":2940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":2941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":2942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":2943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":2944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":2945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":2946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":2947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":2949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":2950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":2951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":2952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":2953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":2954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":2957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":2958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":2959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":2960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":2961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":2962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":2963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":2966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":2967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":2968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":2969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":2970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":2971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":2972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":2973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":2974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":2975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":2976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":2977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":2978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":2979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":2980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":2981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":2982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":2983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":2984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":2985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":2986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":2987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":2988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":2989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":2993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":2994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":2995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":2996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":2997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":2998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":2999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#select-a-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#interactive-cluster","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#sql-warehouse-formerly-sql-endpoint","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-an-oauth-secret-for-the-service-principal","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":3861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":3862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":3863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":3865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":3866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":3867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":3868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":3869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":3870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":3871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":3873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":3875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":3876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":3877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":3878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":3879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":3880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":3881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":3882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":3884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":3885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":3886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":3887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":3888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":3890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":3891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":3892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":3893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":3894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":3895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":3896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":3897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":3898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":3899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":3900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":3902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":3903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":3904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":3905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":3906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":3907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":3908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":3909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":3910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":3911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":3912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":3913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":3914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":3915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":3916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":3917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":3918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":3919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":3920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":3922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":3923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":3924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":3925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":3926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":3927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":3930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":3931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":3932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":3933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":3934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":3936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":3939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":3940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":3941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":3942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":3943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":3944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":3945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":3946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":3947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":3948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":3949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":3950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":3951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":3952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":3953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":3954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":3955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":3956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":3957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":3959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":3960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":3961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":3962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":3968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":3969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":3970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":3971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":3972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":3973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":3974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":3975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":3976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":3977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":3978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":3979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":3980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":3981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":3982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":3984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":3986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":3987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":3988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":3989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":3991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":3992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":3993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":3994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":3995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":3996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":3997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":3998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":3999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#create-a-service-principal-1","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#add-a-service-principal-to-your-account","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#assign-a-service-principal-to-a-workspace","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-enabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#identity-federation-disabled-1","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-to-crawl-metadata","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#system-tables-extraction-method","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":4846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":4848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":4849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":4850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":4851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":4852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":4853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":4854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":4855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":4857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":4858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":4859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":4860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":4861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":4863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":4864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":4866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":4867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":4868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":4869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":4870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":4871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":4872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":4873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":4875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":4876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":4877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":4878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":4879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":4880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":4881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":4882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":4883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":4884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":4885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":4886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":4887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":4888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":4889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":4890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":4891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":4892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":4893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":4895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":4896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":4897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":4898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":4899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":4900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":4903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":4904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":4905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":4906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":4907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":4909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":4912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":4913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":4914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":4915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":4916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":4917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":4918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":4919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":4920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":4921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":4922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":4923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":4924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":4925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":4926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":4927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":4928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":4929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":4930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":4932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":4933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":4934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":4935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":4941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":4942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":4943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":4944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":4945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":4946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":4947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":4948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":4949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":4950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":4951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":4952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":4953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":4954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":4955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":4957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":4959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":4960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":4961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":4962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":4964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":4965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":4966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":4967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":4968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":4969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":4970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":4971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":4972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#cross-workspace-extraction","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":4973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Eliminate the need for separate crawler configurations by using a single service principal to crawl metadata from all workspaces within a Databricks metastore. This guide walks you through configuring the necessary permissions to enable cross-workspace extraction."}
{"id":4974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Cross-workspace extraction isn't supported for REST API or JDBC extraction methods. Before you begin, make sure you have: A Unity Catalog-enabled Databricks workspace Account admin access to create and manage service principals Workspace admin access to grant permissions across all target workspaces At least one active SQL warehouse in each workspace you intend to crawl Set up Databricks authentication completed with one of the supported authentication methods System table extraction enabled for lineage and usage extraction Permissions required The service principal needs the following permissions to enable cross-workspace extraction: CAN_USE on SQL warehouses in each workspace SELECT on system.access.workspace_latest table USE CATALOG , BROWSE , and SELECT on all catalogs you want to crawl Add service principal to all workspaces You must use a single, common service principal that has been granted access to all Databricks workspaces you intend to crawl within the metastore."}
{"id":4975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Log in to your Databricks account console as an account admin From the left menu, click Workspaces and select a workspace From the tabs along the top, click the Permissions tab In the upper right, click Add permissions In the Add permissions dialog: For User, group, or service principal , select your service principal For Permission , select workspace User Click Add For User, group, or service principal , select your service principal For Permission , select workspace User Repeat steps 2-5 for each workspace you intend to crawl Grant permissions Configure the necessary permissions for the service principal to access and extract metadata from all workspaces within the metastore. SQL workspace permissions: The service principal must have usage permissions on at least one active SQL warehouse within each workspace."}
{"id":4976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"The extractor uses the smallest available warehouse to run its discovery queries. Connect to your Databricks workspace using a SQL client or the SQL editor Connect to your Databricks workspace using a SQL client or the SQL editor Run the following command for each workspace, replacing the placeholders: GRANT CAN_USE ON WAREHOUSE TO ` ` ; Replace with your actual warehouse name Replace with your service principal's application ID Example GRANT CAN_USE ON WAREHOUSE production - warehouse TO ` 12345678-1234-1234-1234-123456789012 ` ; Run the following command for each workspace, replacing the placeholders: Replace with your actual warehouse name Replace with your service principal's application ID Log in to your Databricks workspace as a workspace admin From the left menu, click SQL Warehouses On the Compute page, for each SQL warehouse, click the 3-dot icon and then click Permissions Select Can use permission Click Add to assign the permission System table permissions: Access to the system schema is essential for workspace and lineage discovery."}
{"id":4977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Via SQL Via UI Connect to your Databricks workspace using a SQL client or the SQL editor Run the following command, replacing the placeholder: GRANT SELECT ON TABLE system. access."}
{"id":4978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"workspace_latest TO ` ` ; Replace with your service principal's application ID Example GRANT SELECT ON TABLE system. access."}
{"id":4979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"workspace_latest TO ` 12345678-1234-1234-1234-123456789012 ` ; Log in to your Databricks workspace as a workspace admin From the left menu, click Catalog In the Catalog Explorer , navigate to system > access Click on the workspace_latest table Click the Permissions tab and then click Grant In the Grant permissions dialog: Under Principals , select your service principal Under Privileges , check SELECT Click Grant to apply the permissions System table permissions: Access to the system schema is essential for workspace and lineage discovery. Connect to your Databricks workspace using a SQL client or the SQL editor Connect to your Databricks workspace using a SQL client or the SQL editor Run the following command, replacing the placeholder: GRANT SELECT ON TABLE system."}
{"id":4980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"access. workspace_latest TO ` ` ; Replace with your service principal's application ID Example GRANT SELECT ON TABLE system."}
{"id":4981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"access. workspace_latest TO ` 12345678-1234-1234-1234-123456789012 ` ; Run the following command, replacing the placeholder: Replace with your service principal's application ID Log in to your Databricks workspace as a workspace admin From the left menu, click Catalog In the Catalog Explorer , navigate to system > access Click on the workspace_latest table Click the Permissions tab and then click Grant In the Grant permissions dialog: Under Principals , select your service principal Under Privileges , check SELECT Click Grant to apply the permissions Under Principals , select your service principal Under Privileges , check SELECT Click Grant to apply the permissions Asset permissions: The service principal requires permissions to \"see\" and \"read\" the metadata for all data assets you wish to extract."}
{"id":4982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"These grants must be applied to all private, public, and shared catalogs that are in scope for crawling. Important!"}
{"id":4983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"For private catalogs, grant permissions from each workspace. For public catalogs, grant from any workspace."}
{"id":4984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Via SQL Via UI Connect to your Databricks workspace using a SQL client or the SQL editor Grant catalog-level permissions (required even when using BROWSE - BROWSE automatically grants access to all schemas and tables): GRANT USE CATALOG ON CATALOG TO ` ` ; GRANT BROWSE ON CATALOG TO ` ` ; Replace with your actual catalog name Replace with your service principal's application ID If not using BROWSE, along with catalog permissions, grant additional permissions: Grant schema-level permissions: GRANT USE SCHEMA ON SCHEMA. TO ` ` ; Replace and with your actual values Replace with your service principal's application ID Example GRANT USE CATALOG ON CATALOG main TO ` 12345678-1234-1234-1234-123456789012 ` ; GRANT BROWSE ON CATALOG main TO ` 12345678-1234-1234-1234-123456789012 ` ; Log in to your Databricks workspace as a workspace admin From the left menu, click Catalog In the Catalog Explorer , navigate to the catalog you want to grant permissions on (for example, main ) Click the Permissions tab and then click Grant In the Grant permissions dialog: Under Principals , select your service principal Under Privileges , check the following permissions: USE CATALOG USE SCHEMA BROWSE SELECT Click Grant to apply the permissions Repeat steps 3-5 for each catalog you want to crawl in Atlan Asset permissions: The service principal requires permissions to \"see\" and \"read\" the metadata for all data assets you wish to extract."}
{"id":4985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"These grants must be applied to all private, public, and shared catalogs that are in scope for crawling. For private catalogs, grant permissions from each workspace."}
{"id":4986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"For public catalogs, grant from any workspace. Connect to your Databricks workspace using a SQL client or the SQL editor Connect to your Databricks workspace using a SQL client or the SQL editor Grant catalog-level permissions (required even when using BROWSE - BROWSE automatically grants access to all schemas and tables): GRANT USE CATALOG ON CATALOG TO ` ` ; GRANT BROWSE ON CATALOG TO ` ` ; Replace with your actual catalog name Replace with your service principal's application ID Grant catalog-level permissions (required even when using BROWSE - BROWSE automatically grants access to all schemas and tables): Replace with your actual catalog name Replace with your service principal's application ID If not using BROWSE, along with catalog permissions, grant additional permissions: Grant schema-level permissions: GRANT USE SCHEMA ON SCHEMA."}
{"id":4987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"TO ` ` ; Replace and with your actual values Replace with your service principal's application ID If not using BROWSE, along with catalog permissions, grant additional permissions: Grant schema-level permissions: GRANT USE SCHEMA ON SCHEMA. TO ` ` ; Replace and with your actual values Replace with your service principal's application ID Grant schema-level permissions: Replace and with your actual values Replace with your service principal's application ID Log in to your Databricks workspace as a workspace admin From the left menu, click Catalog In the Catalog Explorer , navigate to the catalog you want to grant permissions on (for example, main ) Click the Permissions tab and then click Grant In the Grant permissions dialog: Under Principals , select your service principal Under Privileges , check the following permissions: USE CATALOG USE SCHEMA BROWSE SELECT Click Grant to apply the permissions Under Principals , select your service principal Under Privileges , check the following permissions: Click Grant to apply the permissions Repeat steps 3-5 for each catalog you want to crawl in Atlan Need help?"}
{"id":4988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-cross-workspace-extraction","title":"Set up cross-workspace extraction | Atlan Documentation","text":"Check Troubleshooting Databricks connectivity for common issues Contact Atlan support for help with setup or integration Next steps Add service principal to all workspaces"}
{"id":4989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":4990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":4991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":4992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":4993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":4994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":4995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":4996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":4997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":4998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":4999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-query-and-preview-data","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5128,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"There are two ways to query data in Atlan: writing your own SQL using the Visual Query Builder Atlan pushes all queries to the source (no data is stored in Atlan). In addition, Atlan applies access policies to the results before displaying them."}
{"id":5129,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Write your own SQL Anyone with the knowledge to write SQL. Any Atlan user with data access to the asset can query data."}
{"id":5130,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"To query an asset with your own SQL: From the left menu of any screen, click Insights. Use the Select database dropdown to choose another database, if necessary."}
{"id":5131,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Hover over the table or view, and click the play icon. This writes and runs a basic preview query."}
{"id":5132,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) Click the open asset sidebar icon to view more details in the asset sidebar. (Optional) Click the eye icon to view a preview of the query results."}
{"id":5133,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) Click the 3-dot icon for more options: Click Set editor context to set the same connection, database, and schema name in the query editor as selected in the Explorer tab. Click Place name in editor to view the asset name in the query editor."}
{"id":5134,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click Copy path to copy the full path of the asset, including database and schema names. (Optional) Click the open asset sidebar icon to view more details in the asset sidebar."}
{"id":5135,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) Click the eye icon to view a preview of the query results. (Optional) Click the 3-dot icon for more options: Click Set editor context to set the same connection, database, and schema name in the query editor as selected in the Explorer tab."}
{"id":5136,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click Place name in editor to view the asset name in the query editor. Click Copy path to copy the full path of the asset, including database and schema names."}
{"id":5137,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click Set editor context to set the same connection, database, and schema name in the query editor as selected in the Explorer tab. Click Place name in editor to view the asset name in the query editor."}
{"id":5138,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click Copy path to copy the full path of the asset, including database and schema names. Under the Untitled tab on the right, change the sample query or write your own - separate multiple queries with a semicolon ;."}
{"id":5139,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click the Run button in the upper right to test your query as you write it. (Optional) Click the downward arrow next to the Run button to export query results via email or schedule the query."}
{"id":5140,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) Click the downward arrow next to the Run button to export query results via email or schedule the query. (Optional) If you have multiple tabs open in the query editor, right-click a tab to open the tabs menu."}
{"id":5141,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"You can close a specific tab or all tabs, or duplicate the query. (Optional) From the top right of the query editor, click the 3-dot icon for additional query editor actions or to customize it further: Click or hover over Duplicate query to create a duplicate version of your query."}
{"id":5142,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Open command palette to view the actions you can run inside the query editor. Click or hover over Themes and then select your preferred theme for the query editor."}
{"id":5143,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Tab spacing to change the tab spacing for your queries. Click or hover over Font size to change the font size for your queries."}
{"id":5144,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Cursor to change the cursor position in the query editor. Click or hover over Autosuggestions to turn off autosuggestions for assets in the query editor."}
{"id":5145,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Duplicate query to create a duplicate version of your query. Click or hover over Open command palette to view the actions you can run inside the query editor."}
{"id":5146,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Themes and then select your preferred theme for the query editor. Click or hover over Tab spacing to change the tab spacing for your queries."}
{"id":5147,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Font size to change the font size for your queries. Click or hover over Cursor to change the cursor position in the query editor."}
{"id":5148,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click or hover over Autosuggestions to turn off autosuggestions for assets in the query editor. The editor supports all read-based SQL statements, including JOIN."}
{"id":5149,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"The editor will not run any write-based statements. The following SQL statements are not supported: You can select the context for your query to the left of the Run button."}
{"id":5150,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Then you won't need to fully qualify table names with schema and database names. Use the Visual Query Builder Any Atlan user with data access to the asset."}
{"id":5151,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"No SQL knowledge required! To query an asset using the Visual Query Builder: From the left menu of any screen, click Insights."}
{"id":5152,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"At the top of the screen, to the right of the Untitled tab, click the + button and select New visual query. Under Select from choose the table or view you want to query."}
{"id":5153,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) In the column selector to the right, select the column you want to query. Then develop your query: Click the Run button to run the query and preview its results."}
{"id":5154,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click the blue circular + button to add an action to the query. Repeat these steps until your query is complete."}
{"id":5155,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Click the Run button to run the query and preview its results. Click the blue circular + button to add an action to the query."}
{"id":5156,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Repeat these steps until your query is complete. (Optional) If there are any errors in your query, click Auto fix for Atlan to recommend a fix."}
{"id":5157,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"(Optional) In the query results set, click Copy to copy the query results or click Download to export them. You can learn more about the query builder actions in this example."}
{"id":5158,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/insights\/how-tos\/query-data","title":"query data | Atlan Documentation","text":"Write your own SQL Use the Visual Query Builder"}
{"id":5159,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"Once your connection admins have configured bring your own credentials (BYOC) in Atlan, users will need to provide their own credentials before they can view the sample data in the asset profile. This will help you enforce better governance across your organization."}
{"id":5160,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"Any Atlan user with data access to the asset and their own credentials for the data store. Atlan will display a 100-row sample of the data."}
{"id":5161,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"Use your own credentials to view sample data Atlan supports both basic username and password as well as key pair authentication of your credentials. Atlan also supports SSO authentication."}
{"id":5162,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"To set up your own credentials for viewing sample data: On the Assets page, click on an asset to view its asset profile. In the asset profile, click Sample Data."}
{"id":5163,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"To set up your credentials for viewing the sample data, click Get Started. In the popup window, click Get Started once again to proceed."}
{"id":5164,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"In the User credential setup dialog box, Basic is selected as the default authentication option. Enter the following: For Username , enter the username for the connection."}
{"id":5165,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"For Password , enter the password for that connection. For Role , enter your role for that connection."}
{"id":5166,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"For Warehouse , enter the name of the warehouse. For Username , enter the username for the connection."}
{"id":5167,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"For Password , enter the password for that connection. For Role , enter your role for that connection."}
{"id":5168,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"For Warehouse , enter the name of the warehouse. Click the Test Authentication button to confirm your credentials."}
{"id":5169,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"Once authentication is successful, click Done. You can now view sample data using your own credentials!"}
{"id":5170,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"ð When using the key pair method, you'll need to enter your encrypted private key and the private key password to complete the authentication process. Once you've set up your credentials for viewing sample data, you can also manage your credentials."}
{"id":5171,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/references\/provide-credentials-to-view-sample-data","title":"Provide credentials to view sample data | Atlan Documentation","text":"If your admin has enabled sample data download , you can export sample data in a CSV file. Use your own credentials to view sample data"}
{"id":5172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-import-and-update-tags","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan. Atlan enables you to import your Databricks tags , update your Databricks assets with the imported tags, and push the tag updates back to Databricks: Import tags - crawl Databricks tags from Databricks to Atlan Reverse sync - sync Databricks tag updates from Atlan to Databricks Once you've imported your Databricks tags to Atlan: Your Databricks assets in Atlan will be automatically enriched with their Databricks tags."}
{"id":5312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Imported Databricks tags will be mapped to corresponding Atlan tags through case-insensitive name match - multiple Databricks tags can be matched to a single tag in Atlan. You can also attach Databricks tags , including tag values, to your Databricks assets in Atlan - allowing you to categorize your assets at a more granular level."}
{"id":5313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You can filter your assets by Databricks tags and tag values. You can enable reverse sync to push any tag updates for your Databricks assets back to Databricks - including tag values added to assets in Atlan."}
{"id":5314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Enabling reverse sync will only update existing tags in Databricks. It will neither create nor delete any tags in Databricks."}
{"id":5315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You must have a Unity Catalog-enabled workspace and SQL warehouse configured to import Databricks tags in Atlan. Before you can import tags from and push tag updates to Databricks using personal access token , AWS service principal , or Azure service principal authentication, you will need to do the following: Ensure that you have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Create tags or have existing tags in Databricks. Grant permissions to import tags from and push tag updates to Databricks."}
{"id":5317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Import Databricks tags to Atlan You will need to be an admin user in Atlan to import Databricks tags to Atlan. You will also need to work with your Databricks administrator to grant permissions to import tags from Databricks - you may not have access yourself."}
{"id":5318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You can import your Databricks tags to Atlan through one-way tag sync. The synced Databricks tags will be matched to corresponding tags in Atlan through case-insensitive name match and your Databricks assets will be enriched with their synced tags from Databricks."}
{"id":5319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"To import Databricks tags to Atlan, you can either: Create a new Databricks workflow and configure the crawler to import tags. Modify the crawler's configuration for an existing Databricks workflow to change Import Tags to Yes."}
{"id":5320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"If you subsequently modify the workflow to disable tag import, for any tags already imported, Atlan will preserve those tags. Once the crawler has completed running, tags imported from Databricks will be available to use for tagging assets!"}
{"id":5321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"ð View Databricks tags in Atlan Once you've imported your Databricks tags, you will be able to view and manage your Databricks tags in Atlan. To view Databricks tags: From the left menu of any screen, click Governance."}
{"id":5322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Under the Governance heading of the _Governance cente_r, click Tags. (Optional) Under Tags , click the funnel icon to filter tags by source type."}
{"id":5323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Click Databricks to filter for tags imported from Databricks. From the left menu under Tags , select a synced tag."}
{"id":5324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"(Optional) Click the Linked assets tab to view linked assets for your Databricks tag. (Optional) In the top right, click the pencil icon to add a description and change the tag icon."}
{"id":5325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You cannot rename tags synced from Databricks. Push tag updates to Databricks Any admin or member user in Atlan can configure reverse sync for tag updates to Databricks."}
{"id":5326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"You will also need to work with your Databricks administrator to grant additional permissions to push updates - you may not have access yourself. You can enable reverse sync for your imported Databricks tags in Atlan and push all tag updates for your Databricks assets back to source."}
{"id":5327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Once you have enabled reverse sync, any Databricks assets with tags updated in Atlan will also be updated in Databricks. To enable reverse sync for imported Databricks tags: From the left menu of any screen, click Governance."}
{"id":5328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Under the Governance heading of the _Governance cente_r, click Tags. (Optional) Under Tags , click the funnel icon to filter tags by source type."}
{"id":5329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Click Databricks to filter for tags imported from Databricks. In the left menu under Tags , select a synced Databricks tag - synced tags will display the Databricks icon next to the tag name."}
{"id":5330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Under Synced tags , in the upper right, turn on Enable reverse sync to synchronize tag updates from Atlan to Databricks. In the corresponding confirmation dialog, click Yes, enable it to enable reverse tag sync or click Cancel."}
{"id":5331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Now when you attach Databricks tags to your Databricks assets in Atlan, these tag updates will also be pushed to Databricks! ð Enabling reverse sync will not trigger any updates in Databricks until synced tags are attached to Databricks assets in Atlan."}
{"id":5332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/manage-databricks-tags","title":"Manage Databricks tags | Atlan Documentation","text":"Import Databricks tags to Atlan View Databricks tags in Atlan Push tag updates to Databricks"}
{"id":5333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-extract-lineage-and-usage-from-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Once you have crawled assets from Databricks , you can retrieve lineage from Unity Catalog and usage and popularity metrics from query history or system tables. This is supported for all three authentication methods : personal access token, AWS service principal, and Azure service principal."}
{"id":5473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Both Atlan and Databricks strongly recommend using the system tables method to extract lineage and usage and popularity metrics from Databricks. Usage and popularity metrics can be retrieved for all Databricks users."}
{"id":5474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"However, your Databricks workspace must be Unity Catalog-enabled for the retrieval of lineage and usage and popularity metrics to succeed. You may also need to upgrade existing tables and views to Unity Catalog , as well as reach out to your Databricks account executive to enable lineage in Unity Catalog."}
{"id":5475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"(As of publishing, the feature is still in preview from Databricks on AWS and Azure.) To retrieve lineage and usage from Databricks, rev iew the order of operations and then complete the following steps. Select the extractor To select the Databricks lineage and usage extractor: In the top right of any screen, navigate to New and then click New Workflow."}
{"id":5476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"From the filters along the top, click Miner. From the list of packages, select Databricks Miner and click on Setup Workflow."}
{"id":5477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Configure the lineage extractor Choose your lineage extraction method: In REST API , Atlan connects to your database and extracts lineage directly. In Offline , you will need to first extract lineage yourself and make it available in S3."}
{"id":5478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"In System Table , Atlan connects to your database and queries system tables to extract lineage directly. REST API To configure the Databricks lineage extractor: For Connection , select the connection to extract."}
{"id":5479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"(To select a connection, the crawler must have already run.) Click Next to proceed. Offline extraction method Atlan supports the offline extraction method for extracting lineage from Databricks This method uses Atlan's databricks-extractor tool to extract lineage."}
{"id":5480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"You will need to first extract lineage yourself and make it available in S3. To enter your S3 details: For Connection , select the connection to extract."}
{"id":5481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"(To select a connection, the crawler must have already run.) For Bucket name , enter the name of your S3 bucket. For Bucket prefix , enter the S3 prefix under which all the metadata files exist."}
{"id":5482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"These include extracted-lineage\/result-0.json , extracted-query-history\/result-0.json , and so on. For Bucket region , enter the name of the S3 region."}
{"id":5483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"When complete, at the bottom of the screen, click Next. System table To configure the Databricks lineage extractor: For Connection , select the connection to extract."}
{"id":5484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"(To select a connection, the crawler must have already run.) * Extraction Catalog Type : Default : Select to fetch lineage from the system catalog and access schema. Cloned_catalog : Select to fetch lineage from a cloned catalog and schema."}
{"id":5485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Before proceeding, make sure the following prerequisites are met: You have already created cloned views named column_lineage and table_lineage in your schema. If not, follow the steps in Create cloned views of system tables."}
{"id":5486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"The atlan-user must have SELECT permissions on both views to access lineage data. Then, provide values for the following fields: Cloned Catalog Name Catalog containing the cloned views."}
{"id":5487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned Schema Name Schema containing the cloned views. Default : Select to fetch lineage from the system catalog and access schema."}
{"id":5488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Default : Select to fetch lineage from the system catalog and access schema. Cloned_catalog : Select to fetch lineage from a cloned catalog and schema."}
{"id":5489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Before proceeding, make sure the following prerequisites are met: You have already created cloned views named column_lineage and table_lineage in your schema. If not, follow the steps in Create cloned views of system tables."}
{"id":5490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"The atlan-user must have SELECT permissions on both views to access lineage data. Then, provide values for the following fields: Cloned Catalog Name Catalog containing the cloned views."}
{"id":5491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned Schema Name Schema containing the cloned views. Cloned_catalog : Select to fetch lineage from a cloned catalog and schema."}
{"id":5492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Before proceeding, make sure the following prerequisites are met: You have already created cloned views named column_lineage and table_lineage in your schema. If not, follow the steps in Create cloned views of system tables."}
{"id":5493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"The atlan-user must have SELECT permissions on both views to access lineage data. Then, provide values for the following fields: Cloned Catalog Name Catalog containing the cloned views."}
{"id":5494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned Schema Name Schema containing the cloned views. For SQL Warehouse ID , enter the ID you copied from your SQL warehouse."}
{"id":5495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Click Next to proceed. (Optional) Configure the usage extractor Atlan extracts usage and popularity metrics from: This feature is currently limited to queries on SQL warehouses - queries on interactive clusters are not supported."}
{"id":5496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Additionally, expensive queries and compute costs for Databricks assets are currently unavailable due to limitations of the Databricks APIs. To configure the Databricks usage and popularity extractor: For Fetch Query History and Calculate Popularity , click Yes to retrieve usage and popularity metrics for your Databricks assets."}
{"id":5497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Popularity Extraction Method : Choose one of the following methods to extract usage and popularity metrics:: Click REST API to extract usage and popularity metrics from query history. Click System table to extract metrics directly from system tables: Extraction catalog type for popularity : Choose where to fetch popularity data from: Default : Uses the system catalog and query schema to fetch popularity metrics."}
{"id":5498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned_catalog : Select to fetch popularity from cloned views in a separate catalog and schema. Before proceeding: The query_history view must exist in the provided schema."}
{"id":5499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"The atlan-user must have SELECT permission on the view. Then provide: Cloned Catalog Name The catalog that contains the query_history view."}
{"id":5500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned Schema Name The schema that contains the query_history view. For more information, see Create cloned views of system tables."}
{"id":5501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For SQL Warehouse ID , enter the ID you copied from your SQL warehouse. Click REST API to extract usage and popularity metrics from query history."}
{"id":5502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Click System table to extract metrics directly from system tables: Extraction catalog type for popularity : Choose where to fetch popularity data from: Default : Uses the system catalog and query schema to fetch popularity metrics. Cloned_catalog : Select to fetch popularity from cloned views in a separate catalog and schema."}
{"id":5503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Before proceeding: The query_history view must exist in the provided schema. The atlan-user must have SELECT permission on the view."}
{"id":5504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Then provide: Cloned Catalog Name The catalog that contains the query_history view. Cloned Schema Name The schema that contains the query_history view."}
{"id":5505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For more information, see Create cloned views of system tables. For SQL Warehouse ID , enter the ID you copied from your SQL warehouse."}
{"id":5506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Extraction catalog type for popularity : Choose where to fetch popularity data from: Default : Uses the system catalog and query schema to fetch popularity metrics. Cloned_catalog : Select to fetch popularity from cloned views in a separate catalog and schema."}
{"id":5507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Before proceeding: The query_history view must exist in the provided schema. The atlan-user must have SELECT permission on the view."}
{"id":5508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Then provide: Cloned Catalog Name The catalog that contains the query_history view. Cloned Schema Name The schema that contains the query_history view."}
{"id":5509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For more information, see Create cloned views of system tables. Extraction catalog type for popularity : Choose where to fetch popularity data from: Default : Uses the system catalog and query schema to fetch popularity metrics."}
{"id":5510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned_catalog : Select to fetch popularity from cloned views in a separate catalog and schema. Before proceeding: The query_history view must exist in the provided schema."}
{"id":5511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"The atlan-user must have SELECT permission on the view. Cloned Catalog Name The catalog that contains the query_history view."}
{"id":5512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Cloned Schema Name The schema that contains the query_history view. For more information, see Create cloned views of system tables."}
{"id":5513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For SQL Warehouse ID , enter the ID you copied from your SQL warehouse. For SQL Warehouse ID , enter the ID you copied from your SQL warehouse."}
{"id":5514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Configure the usage extractor: For Popularity Window (days) , 30 days is the maximum limit. You can set a shorter popularity window of less than 30 days."}
{"id":5515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Start time , choose the earliest date from which to mine query history. If you're using the offline extraction method to extract query history from Databricks, skip to the next step."}
{"id":5516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Excluded Users , type the names of users to be excluded while calculating usage metrics for Databricks assets. Press enter after each name to add more names."}
{"id":5517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Popularity Window (days) , 30 days is the maximum limit. You can set a shorter popularity window of less than 30 days."}
{"id":5518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Start time , choose the earliest date from which to mine query history. If you're using the offline extraction method to extract query history from Databricks, skip to the next step."}
{"id":5519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For Excluded Users , type the names of users to be excluded while calculating usage metrics for Databricks assets. Press enter after each name to add more names."}
{"id":5520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"If running the miner for the first time, Atlan recommends setting a start date around three days prior to the current date and then scheduling it daily to build up to two weeks of query history. Mining two weeks of query history on the first miner run may cause delays."}
{"id":5521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"For all subsequent runs, Atlan requires a minimum lag of 24 to 48 hours to capture all the relevant transformations that were part of a session. Learn more about the miner logic here."}
{"id":5522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"Run the extractor To run the Databricks lineage and popularity extractor, after completing the steps above: To check for any permissions or other configuration issues before running the crawler, click Preflight checks. This is currently only supported when using REST API and offline extraction methods."}
{"id":5523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"If you're using system tables, skip to step 2. You can either: To run the crawler once immediately, at the bottom of the screen, click the Run button."}
{"id":5524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button. To run the crawler once immediately, at the bottom of the screen, click the Run button."}
{"id":5525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button. Once the extractor has completed running, you will see lineage for Databricks assets!"}
{"id":5526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/extract-lineage-and-usage-from-databricks","title":"extract lineage and usage from Databricks | Atlan Documentation","text":"ð Select the extractor Configure the lineage extractor (Optional) Configure the usage extractor Run the extractor"}
{"id":5527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#enable-systemaccess-schema","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systeminformation_schematable","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":5861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":5862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":5863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":5864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":5866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":5867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":5868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":5869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":5870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":5871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":5874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":5875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":5876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":5877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":5878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":5880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":5883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":5884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":5885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":5886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":5887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":5888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":5889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":5890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":5891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":5892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":5893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":5894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":5895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":5896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":5897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":5898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":5899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":5900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":5901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":5903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":5904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":5905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":5906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":5912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":5913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":5914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":5915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":5916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":5917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":5918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":5919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":5920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":5921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":5922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":5923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":5924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":5925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":5926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":5928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":5930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":5931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":5932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":5933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":5935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":5936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":5937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":5938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":5939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":5940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":5941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":5942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":5943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":5944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":5945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":5946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":5948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":5949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":5950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":5951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":5952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":5953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":5954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":5956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":5958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":5959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":5960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":5961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":5962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":5963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":5964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":5965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":5967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":5968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":5969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":5970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":5971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":5973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":5974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":5975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":5976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":5977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":5978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":5979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":5980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":5981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":5982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":5983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":5985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":5986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":5987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":5988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":5989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":5990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":5991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":5992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":5993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":5994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":5995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":5996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":5997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":5998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":5999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-enable-systemquery-schema","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6083,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Atlan currently supports usage and popularity metrics for the following connectors: Amazon Redshift - tables, views, and columns. Expensive queries and compute costs for Amazon Redshift assets are currently unavailable due to limitations at source."}
{"id":6084,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Databricks - tables, views, and columns. Expensive queries and compute costs for Databricks assets are currently unavailable due to limitations of the Databricks APIs."}
{"id":6085,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Google BigQuery - tables, views, and columns Microsoft Power BI - reports and dashboards Snowflake - tables, views, and columns Powered by Atlan's enhanced query-mining capabilities, you can view popularity metrics for supported assets: The popularity score of an asset is computed using both the number of queries and the number of users who have queried that asset in the last 30 days. The popularity score of an asset helps determine its relative popularity."}
{"id":6086,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"All assets with a popularity score are then slotted into one of four percentile groups - Least popular , Less popular , Popular , and Most popular. Popularity score is calculated using the following formula: number of distinct users * log (total number of read queries) Time period = 30 days Popularity score is calculated using the following formula: number of distinct users * log (total number of read queries) Popularity score is calculated using the following formula: Time period = 30 days Time period = 30 days The popularity indicator is displayed for all supported assets that have been queried in the last 30 days."}
{"id":6087,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"This indicator visualizes the relative popularity of an asset on a scale of 1 to 4 blue bars - 1 being the lowest score and 4 being the highest. View popularity metrics To view popularity metrics for your assets, complete these steps."}
{"id":6088,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Identify popular assets Being able to identify your most relevant and trusted data assets can help you increase their adoption and drive usage within your organization. To view popularity metrics for an asset: From the left menu in Atlan, click Assets."}
{"id":6089,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"For Connector on the Assets page, select a supported connector - for this example, we'll select Snowflake. From the Popularity sorting menu, click Most popular to view most used assets or Least popular to view least used assets."}
{"id":6090,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Your assets will now have a popularity indicator. To view the popularity popover for an asset, click or hover over the popularity indicator."}
{"id":6091,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"You'll now be able to see all the relevant popularity metrics for your asset! ð View usage metrics in the asset sidebar The new Usage tab in the asset sidebar helps you view usage metadata for your assets."}
{"id":6092,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"For example, if you'd like to appoint a data steward for your data assets, you'll be able to determine the right candidate based on the top users for that asset. You'll also be able to review popular queries or users for a particular table while checking for data compliance."}
{"id":6093,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"To view usage details for an asset: From the left menu in Atlan, click Assets. For Connector on the Assets page, select a supported connector - for this example, we'll select Snowflake."}
{"id":6094,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"From the Popularity sorting menu, click Most popular to view most used assets or Least popular to view least used assets. In the bottom right of any asset card, click or hover over the popularity indicator to open the popularity popover."}
{"id":6095,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"In the popularity popover, click View usage details to view the following: For Usage , view top and recent users in the last 30 days. For Queries , view top five queries by context - Popular , Slow , and Expensive."}
{"id":6096,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Only read queries or SELECT statements are shown for these queries. For Compute , view the total compute cost for an asset."}
{"id":6097,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"The compute cost is split between read and write queries, allowing you to better understand the cost breakdown for individual assets: Read queries - SELECT statements. Write queries - all non- SELECT statements, for example, UPDATE , INSERT , CREATE , and more."}
{"id":6098,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"For Usage , view top and recent users in the last 30 days. For Queries , view top five queries by context - Popular , Slow , and Expensive."}
{"id":6099,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Only read queries or SELECT statements are shown for these queries. For Compute , view the total compute cost for an asset."}
{"id":6100,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"The compute cost is split between read and write queries, allowing you to better understand the cost breakdown for individual assets: Read queries - SELECT statements. Write queries - all non- SELECT statements, for example, UPDATE , INSERT , CREATE , and more."}
{"id":6101,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Read queries - SELECT statements. Write queries - all non- SELECT statements, for example, UPDATE , INSERT , CREATE , and more."}
{"id":6102,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"The usage details for the asset will now appear in the asset sidebar! ð View and sort columns by popularity For any Snowflake, Databricks, or Google BigQuery table or view sorted by popularity, you'll also be able to view and sort the columns by popularity in the asset profile."}
{"id":6103,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"To view column assets by popularity: From the left menu in Atlan, click Assets. For Connector on the Assets page, select a supported connector - for this example, we'll select Snowflake."}
{"id":6104,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"From the Popularity sorting menu, click Most popular to view most used assets or Least popular to view least used assets. Click any asset to open to its asset profile."}
{"id":6105,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"In the Column preview tab of the asset profile, hover over the popularity indicator to view the popularity popover for your columns. You'll now be able to view the popularity score, number of queries and users, and timestamp for last queried for your columns!"}
{"id":6106,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"ð View queries by context Get the context you need before querying an asset to help you optimize your queries. Query popular, slow, or expensive queries from the Usage tab directly in Insights."}
{"id":6107,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"To view and work with queries by context: From the left menu in Atlan, click Assets. For Connector on the Assets page, select a supported connector - for this example, we'll select Snowflake."}
{"id":6108,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"From the Popularity sorting menu, click Most popular to view most used assets or Least popular to view least used assets. In the bottom right of any asset card, click or hover over the popularity indicator to open the popularity popover."}
{"id":6109,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"In the popularity popover, click View usage details. In the Usage tab in the asset sidebar, navigate to Queries and depending on the type of query you'd like to see: Click Popular to see the top five most popular queries."}
{"id":6110,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click Slow to see queries sorted by average duration and last run. Click Expensive to see the top five most expensive queries."}
{"id":6111,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click Popular to see the top five most popular queries. Click Slow to see queries sorted by average duration and last run."}
{"id":6112,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click Expensive to see the top five most expensive queries. Once you've selected the relevant query type, hover over a query card to: Click the expand icon to see the query details."}
{"id":6113,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click the copy icon to copy the query and use it as a template for writing your own queries. Click the code icon to open the query directly in Insights and run it."}
{"id":6114,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click the expand icon to see the query details. Click the copy icon to copy the query and use it as a template for writing your own queries."}
{"id":6115,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"Click the code icon to open the query directly in Insights and run it. If you have any questions about usage and popularity metrics, head over here."}
{"id":6116,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/usage-and-popularity\/how-tos\/interpret-usage-metrics","title":"Interpret usage metrics | Atlan Documentation","text":"View popularity metrics View queries by context"}
{"id":6117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-create-cloned-views-of-system-tables","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-1","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#grant-permissions-2","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6538,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6539,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6540,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6541,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6542,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6543,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6544,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6545,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6546,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6547,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6548,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6549,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6550,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6551,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6552,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6553,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6554,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6555,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6556,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6557,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6558,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6559,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6560,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6561,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6562,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6563,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6564,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6565,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6566,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6567,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6568,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6569,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6570,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6571,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6572,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6573,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6574,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6575,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6576,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6577,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6578,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6579,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6580,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6581,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6582,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6583,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6584,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6585,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6586,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6587,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6588,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6589,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6590,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6591,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6592,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6593,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6594,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6595,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6596,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6597,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6598,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6599,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6600,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6601,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6602,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6603,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6604,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6605,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6606,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6607,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6608,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6609,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6610,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6611,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6612,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6613,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6614,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6615,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6616,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6617,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6618,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6619,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6620,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6621,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6622,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6623,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6624,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6625,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6626,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6627,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6628,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6629,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6630,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6631,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6632,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6633,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6634,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6635,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6636,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6637,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6638,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6639,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6640,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6641,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6642,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6643,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6644,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6645,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6646,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6647,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6648,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6649,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6650,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6651,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6652,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6653,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6654,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6655,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6656,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6657,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6658,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6659,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6660,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6661,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6662,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6663,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6664,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6665,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6666,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6667,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6668,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6669,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6670,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6671,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6672,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#locate-warehouse-id","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6673,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6674,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6675,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6676,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6677,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6678,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6679,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6680,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6681,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6682,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6683,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6684,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6685,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6686,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6687,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6688,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6689,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6690,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6691,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6692,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6693,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6694,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6695,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6696,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6697,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6698,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6699,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6700,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6701,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6702,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6703,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6704,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6705,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6706,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6707,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6708,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6709,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6710,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6711,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6712,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6713,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6714,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6715,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6716,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6717,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6718,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6719,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6720,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6721,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6722,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6723,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6724,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6725,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6726,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6727,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6728,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6729,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6730,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6731,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6732,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6733,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6734,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6735,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6736,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6737,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6738,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6739,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6740,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6741,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6742,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6743,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6744,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6745,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6746,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6747,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6748,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6749,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6750,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6751,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6752,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6753,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6754,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6755,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6756,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6757,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6758,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6759,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6760,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6761,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6762,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6763,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6764,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6765,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6766,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6767,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6768,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6769,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6770,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6771,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6772,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6773,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6774,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6775,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6776,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6777,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6778,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6779,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6780,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6781,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6782,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6783,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6784,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6785,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6786,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6787,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6788,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6789,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6790,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6791,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6792,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6793,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6794,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6795,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6796,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6797,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6798,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6799,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6800,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6801,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6802,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6803,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6804,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6805,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6806,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6807,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6808,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6809,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6810,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6811,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-view-permissions-to-access-databricks-entities-via-apis","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6812,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6813,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6814,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6815,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6816,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6817,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6818,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6819,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6820,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6821,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6822,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6823,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6824,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6825,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6826,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6827,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6828,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6829,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6830,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6831,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6832,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6833,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6834,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6835,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6836,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6837,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6838,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6839,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6840,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6841,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6842,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6843,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6844,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6845,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6846,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6847,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6848,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6849,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6850,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6851,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6852,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6853,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6854,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6855,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6856,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6857,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6858,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6859,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6860,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":6861,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":6862,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":6863,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":6864,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":6865,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6866,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":6867,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":6868,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":6869,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":6870,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":6871,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6872,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":6873,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":6874,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":6875,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":6876,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":6877,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":6878,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6879,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6880,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":6881,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":6882,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":6883,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":6884,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":6885,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6886,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":6887,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6888,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6889,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":6890,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":6891,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":6892,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":6893,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":6894,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":6895,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":6896,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":6897,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":6898,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":6899,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":6900,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":6901,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":6902,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":6903,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":6904,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":6905,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":6906,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":6907,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":6908,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6909,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":6910,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":6911,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":6912,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":6913,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6914,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6915,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6916,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6917,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6918,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":6919,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":6920,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":6921,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":6922,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":6923,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":6924,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":6925,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":6926,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":6927,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":6928,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":6929,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":6930,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":6931,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":6932,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":6933,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6934,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":6935,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6936,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":6937,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":6938,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":6939,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":6940,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6941,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":6942,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":6943,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":6944,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":6945,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":6946,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":6947,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":6948,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":6949,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":6950,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-for-views-and-materialized-views","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":6951,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods: Personal access token authentication AWS service principal authentication Azure service principal authentication Personal access token authentication Check that you have Admin and Databricks SQL access for the Databricks workspace."}
{"id":6952,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"This is required for both cluster options described below. If you don't have this access, contact your Databricks administrator."}
{"id":6953,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Grant user access to workspace To grant workspace access to the user creating a personal access token: From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the user. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6954,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the user to grant access."}
{"id":6955,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the user to grant access."}
{"id":6956,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. Generate a personal access token You can generate a personal access token in your Databricks workspace to the authenticate the integration in Atlan."}
{"id":6957,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To generate a personal access token: From the top right of your Databricks workspace, click your Databricks username, and then from the dropdown, click User Settings. Under the Settings menu, click Developer."}
{"id":6958,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"On the Developer page, next to Access tokens , click Manage. On the Access tokens page, click the Generate new token button."}
{"id":6959,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Generate new token dialog: For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Lifetime (days) , consider removing the number."}
{"id":6960,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"This enables the token to be used indefinitely - it won't need to be refreshed. Important!"}
{"id":6961,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6962,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Comment , enter a description of the token's intended use - for example, Atlan crawler. For Comment , enter a description of the token's intended use - for example, Atlan crawler."}
{"id":6963,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6964,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Important! If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time."}
{"id":6965,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Lifetime (days) , consider removing the number. This enables the token to be used indefinitely - it won't need to be refreshed."}
{"id":6966,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"If you do enter a number, remember that you need to periodically regenerate it and update Atlan's crawler configuration with the new token each time. At the bottom of the dialog, click Generate."}
{"id":6967,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Generate. Copy and save the generated token in a secure location, and then click Done."}
{"id":6968,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Select a cluster Atlan recommends using serverless SQL warehouses for instant compute availability. To enable serverless SQL warehouses, refer to Databricks documentation for AWS Databricks workspaces or Microsoft documentation for Azure Databricks workspaces."}
{"id":6969,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"You can set up personal access token authentication for your Databricks instance using one of the following cluster options: SQL warehouse (formerly SQL endpoint) To confirm an all-purpose interactive cluster is configured: From the left menu of any page of your Databricks instance, click Compute. Under the All-purpose clusters tab, verify you have a cluster defined."}
{"id":6970,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your cluster. Under the Configuration tab, verify the Autopilot options to Terminate after."}
{"id":6971,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"minutes is enabled. At the bottom of the Configuration tab, expand the Advanced options expandable."}
{"id":6972,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6973,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Under the Advanced options expandable, open the JDBC\/ODBC tab. Confirm that all of the fields in this tab are populated, and copy them for use in crawling: Server Hostname , Port , and HTTP Path."}
{"id":6974,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To confirm a SQL warehouse is configured: From the left menu of any page of your Databricks instance, open the dropdown just below the databricks logo and change to SQL. From the refreshed left menu, click SQL Warehouses."}
{"id":6975,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Click the link under the Name column of the table to open your SQL warehouse. Under the Connection details tab, confirm that all of the fields are populated and copy them for use in crawling: Server hostname , Port , and HTTP path."}
{"id":6976,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"AWS service principal authentication You need your AWS Databricks account admin to create a service principal and manage OAuth credentials for the service principal and your AWS Databricks workspace admin to add the service principal to your AWS Databricks workspace - you may not have access yourself. You need the following to authenticate the connection in Atlan: Create a service principal You can create a service principal directly in your Databricks account or from a Databricks workspace."}
{"id":6977,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Identity federation enabled on your workspaces: Databricks recommends creating the service principal in the account and assigning it to workspaces. Identity federation disabled on your workspaces: Databricks recommends that you create your service principal from a workspace."}
{"id":6978,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To create a service principal from your Databricks account, with identify federation enabled: Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6979,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, click Add service principal."}
{"id":6980,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"On the Add service principal page, enter a name for the service principal and then click Add. Once the service principal has been created, you can assign it to your identity federated workspace."}
{"id":6981,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal. From the tabs along the top of your workspace page, click the Permissions tab."}
{"id":6982,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Permissions page, click Add permissions. In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created."}
{"id":6983,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. For User, group, or service principal , select the service principal you created."}
{"id":6984,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For Permission , click the dropdown and select workspace User. To create a service principal from a Databricks workspace, with identity federation disabled: Log in to your AWS Databricks workspace as a workspace admin."}
{"id":6985,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings. In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access."}
{"id":6986,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"On the Identity and access page, under Management and permissions , next to Service principals , click Manage. In the upper right of the Service principals page, click Add service principal."}
{"id":6987,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Add service principal dialog, click the Add new button. For New service principal display name , enter a name for the service principal and then click Add."}
{"id":6988,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Create an OAuth secret for the service principal You need to create an OAuth secret to authenticate to Databricks REST APIs. To create an OAuth secret for the service principal : Log in to your Databricks account console as an account admin."}
{"id":6989,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Log in to your Databricks account console as an account admin. From the left menu of the account console, click User management."}
{"id":6990,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":6991,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of the User management page, click the Service principals tab. In the upper right of the Service principals page, select the service principal you created."}
{"id":6992,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, select the service principal you created. On the service principal page, under OAuth secrets , click Generate secret."}
{"id":6993,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"On the service principal page, under OAuth secrets , click Generate secret. From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location."}
{"id":6994,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"danger Note that this secret is only revealed once during creation. The client ID is the same as the application ID of the service principal."}
{"id":6995,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the Generate secret dialog, copy the Secret and Client ID and store it in a secure location. Note that this secret is only revealed once during creation."}
{"id":6996,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"The client ID is the same as the application ID of the service principal. Once you've copied the client ID and secret, click Done."}
{"id":6997,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Once you've copied the client ID and secret, click Done. Azure service principal authentication You need your Azure Databricks account admin to create a service principal and your Azure Databricks workspace admin to add the service principal to your Azure Databricks workspace - you may not have access yourself."}
{"id":6998,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"You need the following to authenticate the connection in Atlan: Client ID (application ID) Tenant ID (directory ID) Create a service principal To use service principals on Azure Databricks , an admin user must create a new Microsoft Entra ID (formerly Azure Active Directory) application and then add it to the Azure Databricks workspace to use as a service principal. To create a service principal: Sign in to the Azure portal."}
{"id":6999,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to create the service principal. Click**+ Add and select App registration**."}
{"id":7000,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For_Name_, enter a name for the application. In the_Supported account types_section, select Accounts in this organizational directory only (Single tenant) and then click Register."}
{"id":7001,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID To generate a client secret, within_Manage_, click Certificates & secrets. On the_Client secrets_tab, click New client secret."}
{"id":7002,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the_Add a client secret_dialog, enter the following details: For Description , enter a description for the client secret. For_Expires_, select an expiry time period for the client secret and then click Add."}
{"id":7003,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Copy and store the client secret's_Value_in a secure place. Add a service principal to your account To add a service principal to your Azure Databricks account: Log in to your Azure Databricks account console as an account admin."}
{"id":7004,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of the account console, click User management. From the tabs along the top of the User management page, click the Service principals tab."}
{"id":7005,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. On the Add service principal page, enter a name for the service principal."}
{"id":7006,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Under UUID , paste the Application (client) ID for the service principal. Click Add."}
{"id":7007,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Assign a service principal to a workspace To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page."}
{"id":7008,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks account: Log in to your Databricks account console as an account admin. From the left menu of the account console, click Workspaces and then select a workspace to which you want to add the service principal."}
{"id":7009,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the tabs along the top of your workspace page, click the Permissions tab. In the upper right of the Permissions page, click Add permissions."}
{"id":7010,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Add permissions dialog, enter the following details: For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":7011,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For User, group, or service principal , select the service principal you created. For Permission , click the dropdown to select workspace User."}
{"id":7012,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To assign a service principal to your Azure Databricks workspace: Log in to your Azure Databricks workspace as a workspace admin. From the top right of your workspace, click your username, and then from the dropdown, click Admin Settings."}
{"id":7013,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Settings page, under the Workspace admin subheading, click Identity and access. On the Identity and access page, under Management and permissions , next to Service principals , click Manage."}
{"id":7014,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the upper right of the Service principals page, click Add service principal. In the Add service principal dialog, click the Add new button."}
{"id":7015,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For New service principal display name , paste the Application (client) ID for the service principal , enter a display name, and then click Add. Grant permissions to crawl metadata You must have a Unity Catalog-enabled Databricks workspace to crawl metadata in Atlan."}
{"id":7016,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To extract metadata, you can grant the BROWSE privilege , currently in public preview. You no longer require the Data Reader preset that granted the following privileges on objects in the catalog - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT."}
{"id":7017,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to a user or service principal: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":7018,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to crawl in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":7019,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privileges , check the BROWSE privilege."}
{"id":7020,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"At the bottom of the dialog, click Grant. Under Principals , click the dropdown and then select the user or service principal."}
{"id":7021,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Under Privileges , check the BROWSE privilege. At the bottom of the dialog, click Grant."}
{"id":7022,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to crawl in Atlan. System tables extraction method To crawl metadata via system tables, you must have a Unity Catalog-enabled workspace and a configured SQL warehouse."}
{"id":7023,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Follow these steps to extract metadata using system tables: Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Create one of the following authentication methods: Personal access token AWS service principal Azure service principal Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Grant the following privileges to the identity you created: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalogs system.information_schema.schemata system.information_schema.tables system.information_schema.columns system.information_schema.key_column_usage system.information_schema.table_constraints Cross-workspace extraction To crawl metadata from all workspaces within a Databricks metastore using a single connection, see Set up cross-workspace extraction for instructions. (Optional) Grant permissions to query and preview data Atlan currently only supports querying data and viewing sample data preview for the personal access token authentication method."}
{"id":7024,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To grant permissions to query data and preview example data: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":7025,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the left menu of the Catalog Explorer page, select the catalog you want to query and preview data from in Atlan. From the tabs along the top of your workspace page, click the Permissions tab and then click the Grant button."}
{"id":7026,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":7027,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":7028,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Under Principals , click the dropdown and then select the user or service principal. Under Privilege presets , click the dropdown and then click Data Reader to enable read-only access to the catalog."}
{"id":7029,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Doing so automatically selects the following privileges - USE CATALOG , USE SCHEMA , EXECUTE , READ VOLUME , and SELECT. At the bottom of the dialog, click Grant."}
{"id":7030,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Repeat steps 3-5 for each catalog you want to query and preview data from in Atlan. (Optional) Grant permissions to import and update tags To import Databricks tags , you must have a Unity Catalog-enabled workspace and a SQL warehouse configured."}
{"id":7031,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports importing Databricks tags using system tables for all three authentication methods. Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following privileges: CAN_USE on a SQL warehouse USE CATALOG on system catalog USE SCHEMA on system.information_schema SELECT on the following tables: system.information_schema.catalog_tags system.information_schema.schema_tags system.information_schema.table_tags system.information_schema.column_tags To push tags updated for assets in Atlan to Databricks, you need to grant the following privileges : APPLY TAG on the object USE CATALOG on the object's parent catalog USE SCHEMA on the object's parent schema (Optional) Grant permissions to extract lineage and usage from system tables You must have a Unity Catalog-enabled workspace to use system tables."}
{"id":7032,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Atlan supports extracting the following for your Databricks assets using system tables : usage and popularity metrics Enable system.access schema You need your account admin to enable the system.access schema using the SystemSchemas API. This enables Atlan to extract lineage using system tables."}
{"id":7033,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In Atlan, one Databricks connection corresponds to one metastore. Repeat the following process for each metastore in your Databricks environment for which you want to extract lineage."}
{"id":7034,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"To verify that system schemas are enabled for each schema, follow the steps in Databricks documentation : List system schemas using the SystemSchemas API to check the status. If enabled for any given schema, the state is EnableCompleted."}
{"id":7035,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"This confirms that the schema has been enabled for that specific metastore. Atlan can only extract lineage using system tables when the state is marked as EnableCompleted."}
{"id":7036,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.information_schema.table To generate lineage with the target type set as PATH for a table, Atlan uses metadata from system.information_schema.table to resolve table paths and dependencies. To enable this, you must grant the following permissions on the relevant catalog, schema, and tables."}
{"id":7037,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"You must be a metastore admin, have the MANAGE privilege on the object, or be the owner of the catalog, schema, or table to grant these permissions. In Atlan, one Databricks connection corresponds to one metastore."}
{"id":7038,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Repeat the following process for each metastore from which you want to extract lineage. Open Catalog Explorer in your Databricks workspace."}
{"id":7039,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Navigate to the catalog (for example, main ) and then to the appropriate schema (for example, sales ). Click the Permissions tab."}
{"id":7040,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Click Grant. Enter the user or group name (principal)."}
{"id":7041,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Assign the following permissions: USAGE on the catalog USAGE on the schema SELECT on each relevant table USAGE on the catalog USAGE on the schema SELECT on each relevant table Click Grant to apply the changes. These privileges enable Atlan to read table definitions and other metadata from the metastore."}
{"id":7042,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) enable system.query schema This is only required if you also want to extract usage and popularity metrics from Databricks. You need your account admin to enable the system.query schema using the SystemSchemas API."}
{"id":7043,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"This enables Atlan to mine query history using system tables for usage and popularity metrics. To verify that system schemas is enabled for each schema, follow the steps in Databricks documentation."}
{"id":7044,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"If enabled for any given schema, the state is EnableCompleted. ð ª Did you know?"}
{"id":7045,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Can't grant SELECT permissions on the system tables in system.access and system.query? Skip the previous steps and create cloned views in a separate catalog and schema."}
{"id":7046,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"See Create cloned views of system tables. Grant permissions Atlan supports extracting Databricks lineage and usage and popularity metrics using system tables for all three authentication methods."}
{"id":7047,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Once you have created a personal access token , an AWS service principal , or an Azure service principal , you will need to grant the following permissions: CAN_USE on a SQL warehouse USE_CATALOG on system catalog USE SCHEMA on system.access schema USE SCHEMA on system.query schema (tomine query history for usage and popularity metrics) SELECT on the following tables: system.query.history (to mine query history for usage and popularity metrics) system.access.table_lineage system.access.column_lineage system.query.history (to mine query history for usage and popularity metrics) You need to create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":7048,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"ð ª Did you know? Can't grant SELECT permissions on the system tables in system.access and system.query?"}
{"id":7049,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Skip the previous steps and create cloned views in a separate catalog and schema. See Create cloned views of system tables."}
{"id":7050,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Create cloned views of system tables When you don't want to grant access to system tables directly, you can create cloned views to expose lineage and popularity metrics through a separate schema. Follow these steps to set up cloned views: Create a catalog and schema to store cloned views."}
{"id":7051,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create a catalog and schema to store cloned views."}
{"id":7052,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Use meaningful and unique names for example, atlan_cloned_catalog and atlan_cloned_schema. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":7053,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":7054,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":7055,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":7056,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":7057,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Popularity metrics CREATE OR REPLACE VIEW. ."}
{"id":7058,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"query_history AS SELECT * FROM system. query."}
{"id":7059,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"history ; Replace and with the catalog and schema names used in your environment. Create cloned views for the following system tables: Lineage tables CREATE OR REPLACE VIEW."}
{"id":7060,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":". column_lineage AS SELECT * FROM system."}
{"id":7061,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"access. column_lineage ; CREATE OR REPLACE VIEW."}
{"id":7062,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":". table_lineage AS SELECT * FROM system."}
{"id":7063,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"access. table_lineage ; Replace and with the catalog and schema names used in your environment."}
{"id":7064,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Popularity metrics CREATE OR REPLACE VIEW."}
{"id":7065,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":". query_history AS SELECT * FROM system."}
{"id":7066,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"query. history ; Replace and with the catalog and schema names used in your environment."}
{"id":7067,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Replace and with the catalog and schema names used in your environment. Grant the following permissions to enable access to the cloned views: CAN_USE on a SQL warehouse USE CATALOG on the catalog (for example, ) USE SCHEMA and SELECT on the schema (for example,."}
{"id":7068,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":") You must create a Databricks connection in Atlan for each metastore. You can use the hostname of your Unity Catalog-enabled workspace as the Host for the connection."}
{"id":7069,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Locate warehouse ID To extract lineage and usage and popularity metrics using system tables, you will also need the warehouse ID of your SQL warehouse. To locate the warehouse ID: Log in to your Databricks workspace as a workspace admin."}
{"id":7070,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, select the warehouse you want to use."}
{"id":7071,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant view permissions to access Databricks entities via APIs Atlan uses Databricks REST APIs to extract metadata for Notebooks, Queries, Jobs, and Pipelines. This information helps to understand which Databricks enitity was used to create a lineage between assets."}
{"id":7072,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Use the steps below for each object type to grant CAN VIEW permission to the Databricks user or service principal configured in your integration: Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":7073,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Notebook API ( \/api\/2.0\/workspace\/list ): Grant CAN VIEW permission on individual notebooks, or on the workspace folder containing the notebooks, or on the entire workspace. For more information, see Manage Access Control Lists with Folders."}
{"id":7074,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":7075,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Queries API ( \/api\/2.0\/sql\/queries ): Grant CAN VIEW permission on individual queries, or on the workspace folder containing the queries, or on the entire workspace. For more information, see View Queries."}
{"id":7076,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly. Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object."}
{"id":7077,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"For more information, see Control Access to a Job. Job API ( \/api\/2.2\/jobs\/list ): Grant CAN VIEW permission on each job object directly."}
{"id":7078,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Databricks Jobs are distinct from notebooks or files and require permission set directly on the job object. For more information, see Control Access to a Job."}
{"id":7079,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":7080,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Pipeline API ( \/api\/2.0\/pipelines ): Grant CAN VIEW permission on each Delta Live Tables (DLT) pipeline object directly. For more information, see Configure Pipeline Permissions."}
{"id":7081,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions for views and materialized views Atlan requires the following permissions to to extract view definitions from and generate lineagefor views and materialized views: Log in to your Databricks workspace as a workspace admin. From the left menu of your workspace, click Catalog."}
{"id":7082,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Catalog Explorer , select the catalog you want to extract view definitions from and generate lineage for in Atlan. From the tabs at the top, click the Permissions tab, and then click Grant."}
{"id":7083,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"In the Grant on (workspace name) dialog, configure the following: Select the user or service principal under Principals. Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Select the user or service principal under Principals."}
{"id":7084,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Select the following privileges under Privilege presets : USE CATALOG USE SCHEMA SELECT Click Grant to apply the permissions. Repeat steps 3 6 for each catalog you want to crawl in Atlan."}
{"id":7085,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"SELECT permission is required to extract the definitions of views and materialized views. If you prefer not to grant SELECT at the catalog level, you can grant it on individual views and materialized views instead."}
{"id":7086,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"(Optional) Grant permissions to mine query history To mine query history using REST API, you will need to assign the CAN MANAGE permission on your SQL warehouses to the user or service principal. To grant permissions to mine query history: Log in to your Databricks workspace as a workspace admin."}
{"id":7087,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"From the left menu of your workspace, click SQL Warehouses. On the Compute page, for each SQL warehouse you want to mine query history, click the 3-dot icon and then click Permissions."}
{"id":7088,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Expand the Can use permissions dropdown and then select Can manage. This permission enables the service principal to view all queries for the warehouse."}
{"id":7089,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/how-tos\/set-up-databricks#optional-grant-permissions-to-mine-query-history","title":"Set up Databricks | Atlan Documentation","text":"Click Add to assign the CAN MANAGE permission to the service principal. Personal access token authentication AWS service principal authentication Azure service principal authentication Grant permissions to crawl metadata (Optional) Grant permissions to query and preview data (Optional) Grant permissions to import and update tags (Optional) Grant permissions to extract lineage and usage from system tables (Optional) Grant view permissions to access Databricks entities via APIs (Optional) Grant permissions for views and materialized views (Optional) Grant permissions to mine query history"}
{"id":7090,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7091,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7092,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7093,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7094,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7095,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7096,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7097,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7098,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7099,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7100,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7101,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7102,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7103,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7104,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7105,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7106,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7107,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7108,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7109,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7110,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7111,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7112,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7113,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7114,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7115,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7116,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7117,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7118,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7119,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7120,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7121,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#__docusaurus_skipToContent_fallback","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7122,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7123,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7124,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7125,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7126,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7127,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7128,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7129,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7130,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7131,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7132,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7133,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7134,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7135,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7136,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7137,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7138,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7139,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7140,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7141,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7142,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7143,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7144,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7145,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7146,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7147,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7148,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7149,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7150,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7151,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7152,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7153,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#before-you-begin","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7154,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7155,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7156,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7157,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7158,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7159,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7160,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7161,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7162,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7163,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7164,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7165,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7166,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7167,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7168,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7169,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7170,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7171,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7172,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7173,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7174,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7175,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7176,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7177,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7178,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7179,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7180,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7181,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7182,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7183,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7184,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7185,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#register-application-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7186,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7187,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7188,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7189,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7190,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7191,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7192,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7193,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7194,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7195,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7196,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7197,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7198,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7199,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7200,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7201,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7202,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7203,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7204,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7205,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7206,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7207,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7208,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7209,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7210,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7211,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7212,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7213,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7214,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7215,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7216,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7217,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#create-security-group-in-microsoft-entra-id","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7218,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7219,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7220,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7221,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7222,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7223,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7224,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7225,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7226,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7227,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7228,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7229,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7230,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7231,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7232,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7233,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7234,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7235,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7236,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7237,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7238,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7239,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7240,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7241,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7242,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7243,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7244,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7245,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7246,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7247,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7248,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7249,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-authentication-options","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7250,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7251,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7252,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7253,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7254,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7255,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7256,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7257,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7258,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7259,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7260,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7261,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7262,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7263,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7264,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7265,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7266,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7267,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7268,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7269,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7270,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7271,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7272,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7273,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7274,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7275,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7276,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7277,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7278,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7279,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7280,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7281,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#service-principal-authentication-recommended","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7282,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7283,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7284,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7285,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7286,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7287,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7288,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7289,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7290,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7291,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7292,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7293,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7294,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7295,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7296,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7297,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7298,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7299,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7300,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7301,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7302,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7303,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7304,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7305,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7306,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7307,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7308,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7309,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7310,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7311,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7312,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7313,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-only","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7314,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7315,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7316,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7317,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7318,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7319,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7320,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7321,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7322,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7323,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7324,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7325,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7326,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7327,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7328,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7329,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7330,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7331,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7332,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7333,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7334,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7335,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7336,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-and-non-admin-apis","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#assign-security-group-to-power-bi-workspaces-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#configure-admin-and-non-admin-api-access-in-powerbi-service-portal","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#delegated-user-authentication","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7458,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7459,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7460,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7461,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7462,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7463,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7464,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7465,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7466,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7467,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7468,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7469,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7470,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7471,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7472,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7473,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#fabric-administrator-role-assignment","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7474,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7475,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7476,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7477,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7478,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7479,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7480,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7481,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7482,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7483,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7484,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7485,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7486,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7487,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7488,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7489,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7490,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7491,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7492,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7493,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7494,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7495,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7496,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7497,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7498,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7499,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7500,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7501,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7502,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7503,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7504,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7505,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#api-permissions","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7506,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Depending on the authentication method you choose, you may need a combination of your Cloud Application Administrator or Application Administrator for Microsoft Entra ID, Microsoft 365 administrator for Microsoft 365, and Fabric Administrator ( formerly known as Power BI Administrator ) for Microsoft Power BI to complete these tasks -> you may not have access yourself. This guide outlines how to set up Microsoft Power BI so it can connect with Atlan for metadata extraction and lineage tracking."}
{"id":7507,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Before you begin Register application in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps > you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization."}
{"id":7508,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To register a new application in Microsoft Entra ID: Log in to the Azure portal. Click App registrations from the left menu."}
{"id":7509,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click + New registration. Enter a name for your client application and click Register."}
{"id":7510,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Application (client) ID Directory (tenant) ID Click Certificates & secrets from the left menu. Under Client secrets , click + New client secret."}
{"id":7511,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter a description, select an expiry time, and click Add. Copy and securely store the client secret Value."}
{"id":7512,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Create security group in Microsoft Entra ID You need your Cloud Application Administrator or Application Administrator to complete these steps - you may not have access yourself. To create a security group for your application: Log in to the Azure portal."}
{"id":7513,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Groups under the Manage section. Click New group."}
{"id":7514,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Set the Group type to Security. Enter a Group name and optional description."}
{"id":7515,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click No members selected. Click Select and then Create."}
{"id":7516,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"By the end of these steps, you have registered an application with Microsoft Entra ID and created a Security Group with the appropriate member. Configure authentication options Atlan supports two authentication methods for fetching metadata from Microsoft Power BI: Service principal authentication (recommended) When using Service Principal authentication, you must decide how the connector shall access metadata to catalog assets and build lineage."}
{"id":7517,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"There are two supported options: This option grants permissions that let the service principal to access only admin-level Power BI APIs. In this mode, Atlan extracts metadata exclusively using administrative endpoints."}
{"id":7518,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"This option is recommended for stricter access control environments. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7519,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To configure admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7520,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group This option grants permissions that let the service principal to access both admin and non-admin Power BI APIs. This enables Atlan to extract richer metadata and build detailed lineage across Power BI assets."}
{"id":7521,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need to be at least a member of the Microsoft Power BI workspace to which you want to add the security group to complete these steps - you may not have access yourself. Make sure that you add the security group from the homepage and not the admin portal."}
{"id":7522,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To assign a Microsoft Power BI workspace role to the security group: Open the Microsoft Power BI homepage. Open Workspaces and select the workspace you want to access from Atlan."}
{"id":7523,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Access. In the panel: Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add."}
{"id":7524,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Enter the name of your security group where it says Enter email addresses Choose one of the following roles: Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Viewer : For workspaces without parameters Contributor : For workspaces with semantic models containing parameters or to generate lineage for measures Member : To generate lineage for dataflows Click Add. You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks - you may not have access yourself."}
{"id":7525,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable both admin and non-admin API access: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7526,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Developer settings : Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Expand Service principals can use Fabric APIs and set to Enabled Add your security group under Specific security groups Click Apply Add your security group under Specific security groups Under Admin API settings : Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Expand Enable service principals to use read-only Power BI admin APIs and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply Add your security group After making these changes, you typically need to wait 15-30 minutes for the settings to take effect across Microsoft's services. Delegated user authentication Atlan doesn't recommend using delegated user authentication as it's also not recommended by Microsoft."}
{"id":7527,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Microsoft 365 administrator to complete these steps - you may not have access yourself. To assign the delegated user to the Fabric Administrator role: Open the Microsoft 365 admin portal."}
{"id":7528,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Users and then Active users from the left menu. Select the delegated user."}
{"id":7529,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Roles , click Manage roles. Expand Show all by category."}
{"id":7530,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Collaboration , select Fabric Administrator. Click Save changes."}
{"id":7531,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"You need your Cloud Application Administrator or Application Administrator to complete these steps, you may not have access yourself. The following permissions are only required for delegated user authentication."}
{"id":7532,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"If using service principal authentication, you don't need to configure any delegated permissions for a service principal it's recommended that you avoid adding these permissions. These are never used and can cause errors that may be hard to troubleshoot."}
{"id":7533,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To add permissions for the registered application : In your app registration, click API permissions under the Manage section. Click Add a permission."}
{"id":7534,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Click Delegated permissions and select: Capacity.Read.All Dataflow.Read.All Dataset.Read.All Report.Read.All Tenant.Read.All Workspace.Read.All Click Grant Admin consent (If you only see the Add permissions button, you aren't an administrator). You need your Fabric Administrator ( formerly known as Power BI Administrator ) to complete these tasks, you may not have access yourself."}
{"id":7535,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"To enable the Microsoft Power BI admin API: Log in to the Power BI admin portal. Click Tenant settings under Admin portal."}
{"id":7536,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Under Admin API settings : Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply. Expand Enhance admin APIs responses with detailed metadata and set to Enabled Add your security group Click Apply Add your security group Expand Enhance admin APIs responses with DAX and mashup expressions and set to Enabled Add your security group Click Apply."}
{"id":7537,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/how-tos\/set-up-microsoft-power-bi#admin-api-settings-configuration","title":"Set up Microsoft Power BI | Atlan Documentation","text":"Add your security group Click Apply. Before you begin Configure authentication options"}
{"id":7538,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7539,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7540,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7541,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7542,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7543,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7544,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7545,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7546,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7547,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7548,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7549,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7550,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7551,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7552,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7553,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7554,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7555,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7556,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7557,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7558,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7559,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7560,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#__docusaurus_skipToContent_fallback","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7561,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7562,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7563,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7564,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7565,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7566,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7567,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7568,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7569,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7570,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7571,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7572,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7573,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7574,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7575,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7576,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7577,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7578,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7579,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7580,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7581,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7582,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7583,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#amazon-web-services-aws","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7584,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7585,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7586,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7587,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7588,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7589,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7590,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7591,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7592,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7593,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7594,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7595,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7596,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7597,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7598,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7599,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7600,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7601,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7602,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7603,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7604,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7605,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7606,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#microsoft-azure","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7607,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7608,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7609,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7610,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7611,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7612,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7613,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7614,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7615,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7616,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7617,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7618,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7619,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7620,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7621,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7622,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7623,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7624,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7625,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7626,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7627,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7628,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7629,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#google-cloud-platform-gcp","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7630,"url":"https:\/\/docs.atlan.com\/platform\/references\/how-are-resources-isolated","title":"How are resources isolated? | Atlan Documentation","text":"Each Atlan customer has their own isolated set of nodes within Kubernetes. The underlying Kubernetes control plane and networking layer (coredns) are shared between tenants."}
{"id":7631,"url":"https:\/\/docs.atlan.com\/platform\/references\/how-are-resources-isolated","title":"How are resources isolated? | Atlan Documentation","text":"To achieve logical isolation, Loft s virtual clusters are implemented. The underlying Kubernetes control plane and networking layer (coredns) are shared between tenants."}
{"id":7632,"url":"https:\/\/docs.atlan.com\/platform\/references\/how-are-resources-isolated","title":"How are resources isolated? | Atlan Documentation","text":"To achieve logical isolation, Loft s virtual clusters are implemented. The compute resources (nodes, nodegroups) and storage are physically isolated between tenants."}
{"id":7633,"url":"https:\/\/docs.atlan.com\/platform\/references\/how-are-resources-isolated","title":"How are resources isolated? | Atlan Documentation","text":"The compute resources (nodes, nodegroups) and storage are physically isolated between tenants. Only Atlan's cloud team is able to manage the AWS , Azure , and GCP resources across these levels of isolation."}
{"id":7634,"url":"https:\/\/docs.atlan.com\/platform\/references\/how-are-resources-isolated","title":"How are resources isolated? | Atlan Documentation","text":"Only Atlan's cloud team is able to manage the AWS , Azure , and GCP resources across these levels of isolation. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7635,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7636,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7637,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7638,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7639,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7640,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7641,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7642,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7643,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7644,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7645,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7646,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7647,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7648,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7649,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7650,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7651,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7652,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7653,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7654,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7655,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7656,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7657,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-components","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7658,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7659,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7660,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7661,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7662,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7663,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7664,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7665,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7666,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7667,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7668,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7669,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7670,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7671,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7672,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7673,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7674,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7675,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7676,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7677,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7678,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7679,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7680,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#platform-management-components","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7681,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7682,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7683,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7684,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7685,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7686,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7687,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7688,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7689,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7690,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7691,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7692,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7693,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7694,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7695,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7696,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7697,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7698,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7699,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7700,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7701,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7702,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7703,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#central-components","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7704,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Atlan is a cloud-first solution. Single-tenant SaaS is the recommended deployment model."}
{"id":7705,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Atlan currently supports hosting tenants on the following cloud platforms: Amazon Web Services (AWS) Microsoft Azure Google Cloud Platform (GCP) The components of Atlan are isolated, across both compute and data. For more details, see How are resources isolated?"}
{"id":7706,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Platform components Kong is an API gateway. It handles rate limiting and token verification on all incoming API requests."}
{"id":7707,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Apache Keycloak is an identity and access management component. It manages everything to do with users, login, SSO and so on."}
{"id":7708,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Heracles is Atlan's API service. It houses the business logic used by the frontend and APIs to interact with other platform components."}
{"id":7709,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"PostgreSQL is a SQL database. Many services on the platform use it for storage."}
{"id":7710,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"HashiCorp Vault is a secret manager. It stores sensitive credentials provided by the user."}
{"id":7711,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Apache Ranger is the policy engine. It provides fine-grained access control over data in the metastore."}
{"id":7712,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Argo Workflows is a workflow orchestrator for k8s. It runs and manages long-running jobs in a container and k8s-native fashion."}
{"id":7713,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Admission Controller is a k8s admission controller. It performs certain actions when Argo Workflows are updated such as workflow alerts."}
{"id":7714,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Apache Zookeeper manages consensus and coordination for the metastore services. Apache Cassandra is an object-oriented database used to store the metastore's data."}
{"id":7715,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Apache Kafka is an event stream. It enables event-driven use cases across the platform."}
{"id":7716,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Heka is Atlan's SQL component. It parses, rewrites and optimizes SQL queries and is powered by Apache Calcite."}
{"id":7717,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Redis is a cache layer used by Heracles. Platform management components Velero performs cluster backups."}
{"id":7718,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Fluent Bit is a logging and metrics processor. It parses and pushes logs from pods to various destinations."}
{"id":7719,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Central components Zenduty is used for incident response. Alerts are sent when something goes wrong in one of the clusters."}
{"id":7720,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Argo CD is used for continuous deployment. Changes in git repositories lead to upgrades in the clusters."}
{"id":7721,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Github Actions update the Docker container images as part of the development process. Sendgrid is used to send emails."}
{"id":7722,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"The frontend is a Vue.js web application that's hosted on S3 and delivered via Amazon CloudFront content delivery network (CDN) service. Alertmanager sends alerts generated by metrics stored in Prometheus."}
{"id":7723,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Grafana provides observability dashboards. VictoriaMetrics is a fast, cost-effective, and scalable monitoring solution and time series database."}
{"id":7724,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"It processes high volumes of data and enables long-term storing. Atlan marketplace (not pictured) The marketplace offers packages (workflows) that perform long-running tasks on the Atlan platform."}
{"id":7725,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"The ecosystem enables the creation of metadata and lineage connectors. See security.atlan.com for the latest policies and standards, reports and certifications, architecture, diagrams and more."}
{"id":7726,"url":"https:\/\/docs.atlan.com\/platform\/references\/atlan-architecture#atlan-marketplace-not-pictured","title":"Atlan architecture | Atlan Documentation","text":"Platform management components Atlan marketplace (not pictured)"}
{"id":7727,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":7728,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":7729,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":7730,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7731,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7732,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":7733,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":7734,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":7735,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":7736,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":7737,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":7738,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":7739,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7740,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7741,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7742,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7743,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":7744,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":7745,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":7746,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":7747,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":7748,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7749,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":7750,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7751,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":7752,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":7753,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":7754,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":7755,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":7756,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":7757,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":7758,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":7759,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":7760,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":7761,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":7762,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":7763,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":7764,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":7765,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":7766,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":7767,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":7768,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7769,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7770,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7771,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7772,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":7773,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":7774,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":7775,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":7776,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":7777,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":7778,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":7779,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#__docusaurus_skipToContent_fallback","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":7780,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":7781,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":7782,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":7783,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7784,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7785,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":7786,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":7787,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":7788,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":7789,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":7790,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":7791,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":7792,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7793,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7794,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7795,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7796,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":7797,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":7798,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":7799,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":7800,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":7801,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7802,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":7803,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7804,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":7805,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":7806,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":7807,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":7808,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":7809,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":7810,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":7811,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":7812,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":7813,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":7814,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":7815,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":7816,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":7817,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":7818,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":7819,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":7820,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":7821,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7822,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7823,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7824,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7825,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":7826,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":7827,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":7828,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":7829,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":7830,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":7831,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":7832,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#supported-tools","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":7833,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":7834,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":7835,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":7836,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7837,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7838,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":7839,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":7840,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":7841,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":7842,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":7843,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":7844,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":7845,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7846,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7847,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7848,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7849,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":7850,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":7851,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":7852,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":7853,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":7854,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7855,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":7856,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7857,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":7858,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":7859,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":7860,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":7861,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":7862,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":7863,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":7864,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":7865,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":7866,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":7867,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":7868,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":7869,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":7870,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":7871,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":7872,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":7873,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":7874,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7875,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7876,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7877,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7878,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":7879,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":7880,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":7881,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":7882,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":7883,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":7884,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":7885,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#install-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":7886,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If you're using managed browsers, you can install and configure the Atlan browser extension for all users in your organization. To do so, you will need to bulk install the extension and deploy a configuration script."}
{"id":7887,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Atlan supports managing the Atlan browser extension for the following: Operating systems: macOS and Microsoft Windows Browsers: Google Chrome and Microsoft Edge The deployment scripts - .mobileconfig file for macOS and PowerShell script for Microsoft Windows - are designed to make only the most necessary modifications required for the Atlan browser extension to function properly. Both deployment methods adhere to the principle of least privilege: The .mobileconfig file for macOS only includes the configuration settings required to install and operate the Atlan browser extension."}
{"id":7888,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The PowerShell script creates essential registry keys required for the Atlan browser extension to operate on Microsoft Windows systems. To configure the Atlan browser extension for a managed browser, you must complete these steps in the following order: Configure the browser extension Bulk install the browser extension Deploy the configuration script (Optional) Verify and monitor the installation Configure the browser extension You will need to be an admin in Atlan to configure the browser extension for users in your organization."}
{"id":7889,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"You will also need inputs and approval from the IT administrator of your organization. You can configure the browser extension and then download a configuration script to bulk install and deploy it for everyone in your organization."}
{"id":7890,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"To configure the browser extension, from within Atlan: From the left menu on any screen, click Admin. Under Workspace , click Integrations."}
{"id":7891,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Bulk install the browser extension , click the Set up now button."}
{"id":7892,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"In the Set up browser extension form, enter the following details: For Choose browser , the browser and operating system values will be prefilled based on what you're currently using - you can modify the fields, if required. For Your Atlan domain , enter the URL of your Atlan instance - for example, https:\/\/(instance_name).atlan.com."}
{"id":7893,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"info ð ª Did you know? If you enable multiple Atlan domains, your users will be able to select the most relevant Atlan domain from a dropdown menu while using the browser extension."}
{"id":7894,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The default value in the dropdown will be the Atlan instance entered as Your Atlan domain. If your organization does not have multiple Atlan domains, only the default selection will be displayed."}
{"id":7895,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"(Optional) For Advanced settings , you can configure the following: If you have multiple Atlan instances, toggle on Multiple Atlan domains and then enter the URLs of your additional Atlan instances. Click + Add to add more Atlan domains."}
{"id":7896,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, toggle on Custom data source domain. Click + Add to add more custom domains for data sources."}
{"id":7897,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of your custom data source domain."}
{"id":7898,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"info ð ª Did you know? For any supported tools configured while setting up the managed browser extension, your users will not be able to update or remove these selections."}
{"id":7899,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. Click the Download Script button to download the corresponding configuration script."}
{"id":7900,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The IT administrator(s) in your organization will need to install this configuration file in your organization's devices using a mobile device management (MDM) software. Administrative permissions to the MDM platform are required to complete the setup."}
{"id":7901,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Based on your operating system, the downloaded file can be one of the following two types: .mobileconfig - use this file to configure profiles with specific settings in macOS devices. .ps1 - use this PowerShell script to create registry keys in Microsoft Windows devices."}
{"id":7902,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Choose browser , the browser and operating system values will be prefilled based on what you're currently using - you can modify the fields, if required. For Choose browser , the browser and operating system values will be prefilled based on what you're currently using - you can modify the fields, if required."}
{"id":7903,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Your Atlan domain , enter the URL of your Atlan instance - for example, https:\/\/(instance_name).atlan.com. info ð ª Did you know?"}
{"id":7904,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If you enable multiple Atlan domains, your users will be able to select the most relevant Atlan domain from a dropdown menu while using the browser extension. The default value in the dropdown will be the Atlan instance entered as Your Atlan domain."}
{"id":7905,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If your organization does not have multiple Atlan domains, only the default selection will be displayed. For Your Atlan domain , enter the URL of your Atlan instance - for example, https:\/\/(instance_name).atlan.com."}
{"id":7906,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"ð ª Did you know? If you enable multiple Atlan domains, your users will be able to select the most relevant Atlan domain from a dropdown menu while using the browser extension."}
{"id":7907,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The default value in the dropdown will be the Atlan instance entered as Your Atlan domain. If your organization does not have multiple Atlan domains, only the default selection will be displayed."}
{"id":7908,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"(Optional) For Advanced settings , you can configure the following: If you have multiple Atlan instances, toggle on Multiple Atlan domains and then enter the URLs of your additional Atlan instances. Click + Add to add more Atlan domains."}
{"id":7909,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, toggle on Custom data source domain. Click + Add to add more custom domains for data sources."}
{"id":7910,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of your custom data source domain."}
{"id":7911,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"info ð ª Did you know? For any supported tools configured while setting up the managed browser extension, your users will not be able to update or remove these selections."}
{"id":7912,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Advanced settings , you can configure the following: If you have multiple Atlan instances, toggle on Multiple Atlan domains and then enter the URLs of your additional Atlan instances."}
{"id":7913,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Click + Add to add more Atlan domains. If you have multiple Atlan instances, toggle on Multiple Atlan domains and then enter the URLs of your additional Atlan instances."}
{"id":7914,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Click + Add to add more Atlan domains. If your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, toggle on Custom data source domain."}
{"id":7915,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Click + Add to add more custom domains for data sources. For Connector , select a supported tool for the browser extension."}
{"id":7916,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"In the adjacent field, enter the URL of your custom data source domain. info ð ª Did you know?"}
{"id":7917,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For any supported tools configured while setting up the managed browser extension, your users will not be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":7918,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"If your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, toggle on Custom data source domain. Click + Add to add more custom domains for data sources."}
{"id":7919,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of your custom data source domain."}
{"id":7920,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"ð ª Did you know? For any supported tools configured while setting up the managed browser extension, your users will not be able to update or remove these selections."}
{"id":7921,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. Click the Download Script button to download the corresponding configuration script."}
{"id":7922,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The IT administrator(s) in your organization will need to install this configuration file in your organization's devices using a mobile device management (MDM) software. Administrative permissions to the MDM platform are required to complete the setup."}
{"id":7923,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Based on your operating system, the downloaded file can be one of the following two types: .mobileconfig - use this file to configure profiles with specific settings in macOS devices. .ps1 - use this PowerShell script to create registry keys in Microsoft Windows devices."}
{"id":7924,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Click the Download Script button to download the corresponding configuration script. The IT administrator(s) in your organization will need to install this configuration file in your organization's devices using a mobile device management (MDM) software."}
{"id":7925,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Administrative permissions to the MDM platform are required to complete the setup. Based on your operating system, the downloaded file can be one of the following two types: .mobileconfig - use this file to configure profiles with specific settings in macOS devices."}
{"id":7926,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":".ps1 - use this PowerShell script to create registry keys in Microsoft Windows devices. Bulk install the browser extension You will need to have administrator access to your organization's mobile device management (MDM) software with the permission to add and deploy new policies to all users."}
{"id":7927,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"You will also need inputs and approval from your Atlan admin. You will need to configure the ExtensionInstallForcelist browser policy for either Google Chrome or Microsoft Edge to force-install the extension for everyone in your organization."}
{"id":7928,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The ExtensionInstallForcelist browser policy: Governs extensions that can be silently installed and automatically enabled for all users. Provides extension IDs that the browser will automatically install and enable when a user logs in."}
{"id":7929,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Google Chrome To bulk install the Atlan browser extension in Google Chrome, follow the steps in Google documentation: Force install apps and extensions. Microsoft Edge To bulk install the Atlan browser extension in Microsoft Edge, follow the steps in Microsoft documentation: Force-install an extension."}
{"id":7930,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For the Extension\/App IDs and update URLs to be silently installed (Device) field, copy and paste the following value: fipjfjlalpnbejlmmpfnmlkadjgaaheg is the extension-id for the Atlan browser extension. Deploy the configuration script You will need to have administrator access to your organization's mobile device management (MDM) software with the permission to add and deploy new policies to all users."}
{"id":7931,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"You will also need inputs and approval from your Atlan admin. The browser extension relies on managed storage for configuring domains in the Atlan extension."}
{"id":7932,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"The values for managed storage can be configured through: A configuration profile in macOS Registry keys in Microsoft Windows Although Atlan's solution is platform-agnostic, the following example pertains to Microsoft Intune. You will need to create a custom managed profile to configure the domains for the Atlan browser extension."}
{"id":7933,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"To deploy the .mobileconfig file for your organization, you can use any MDM platform. Microsoft Intune - follow the steps in Custom configuration profile settings."}
{"id":7934,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Microsoft Windows You will need to create registry keys to deploy the extension. You can create the required registry keys with a PowerShell script, which can then be deployed to your users devices using an MDM software."}
{"id":7935,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"To deploy the PowerShell configuration script for your organization, you can use any MDM platform. Microsoft Intune - follow the steps in Create a script policy and assign it."}
{"id":7936,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"For Script settings , enter the following details: Script location - upload the .ps1 configuration script downloaded from Atlan. Run this script using the logged on credentials - change to No."}
{"id":7937,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Enforce script signature check - change to No. Run script in 64 bit PowerShell Host - change to Yes."}
{"id":7938,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Script location - upload the .ps1 configuration script downloaded from Atlan. Run this script using the logged on credentials - change to No."}
{"id":7939,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Enforce script signature check - change to No. Run script in 64 bit PowerShell Host - change to Yes."}
{"id":7940,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Verify and monitor the installation To ensure that the Atlan browser extension has been successfully deployed across all selected devices in your organization, you can: Verify the installation - after you have deployed the policies, check a few target devices to ensure that the extension was installed and configured correctly. Monitor compliance - monitor the compliance status of the policy and troubleshoot any issues."}
{"id":7941,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Your users will now be able to use the Atlan browser extension in a managed browser! ð Once the managed browser has synced with the latest configuration changes for your organization, the Atlan browser extension will be automatically installed and a new tab will open to indicate that the Atlan browser extension is now active."}
{"id":7942,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/configure-the-extension-for-managed-browsers","title":"Configure the extension for managed browsers | Atlan Documentation","text":"Configure the browser extension Bulk install the browser extension Deploy the configuration script Verify and monitor the installation"}
{"id":7943,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":7944,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":7945,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":7946,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7947,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":7948,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":7949,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":7950,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":7951,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":7952,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":7953,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":7954,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":7955,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7956,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7957,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":7958,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":7959,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":7960,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":7961,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":7962,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":7963,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":7964,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7965,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":7966,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":7967,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":7968,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":7969,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":7970,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":7971,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":7972,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":7973,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":7974,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":7975,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":7976,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":7977,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":7978,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":7979,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":7980,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":7981,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":7982,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":7983,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":7984,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7985,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":7986,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7987,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":7988,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":7989,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":7990,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":7991,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":7992,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":7993,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":7994,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":7995,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":7996,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":7997,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":7998,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":7999,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8000,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8001,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8002,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8003,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8004,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8005,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8006,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8007,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8008,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8009,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8010,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8011,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8012,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8013,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8014,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8015,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8016,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8017,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8018,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8019,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8020,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8021,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8022,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8023,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8024,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8025,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8026,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8027,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8028,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8029,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8030,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8031,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8032,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8033,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8034,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8035,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8036,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8037,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8038,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8039,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8040,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8041,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8042,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8043,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8044,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8045,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8046,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8047,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8048,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#optional-configure-custom-domains-as-an-admin","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8049,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":8050,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":8051,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":8052,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8053,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8054,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8055,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8056,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8057,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8058,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8059,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8060,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8061,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8062,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8063,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8064,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8065,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8066,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8067,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8068,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8069,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8070,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8071,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8072,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8073,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8074,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8075,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8076,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8077,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8078,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8079,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8080,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8081,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8082,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8083,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8084,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8085,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8086,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8087,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8088,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8089,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8090,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8091,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8092,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8093,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8094,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8095,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8096,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8097,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8098,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8099,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8100,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8101,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#configure-the-extension-as-a-user","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8102,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":8103,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":8104,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":8105,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8106,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8107,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8108,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8109,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8110,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8111,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8112,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8113,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8114,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8115,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8116,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8117,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8118,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8119,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8120,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8121,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8122,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8123,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8124,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8125,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8126,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8127,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8128,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8129,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8130,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8131,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8132,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8133,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8134,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8135,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8136,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8137,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8138,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8139,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8140,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8141,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8142,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8143,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8144,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8145,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8146,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8147,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8148,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8149,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8150,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8151,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8152,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8153,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8154,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#usage","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8155,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"Atlan adheres to strict security standards for the browser extension. Atlan mandates security throughout the extension coding lifecycle: Hardening configurations through content security policies, Validating all inputs, Requiring least privileges, Employing defense-in-depth techniques like code obfuscation to frustrate reverse engineering, Accessing customer resources over secure HTTPS channels after SSL certificate verification to prevent tampering."}
{"id":8156,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"Atlan follows proven CI\/CD methodologies used for our SaaS application, enabling rapid and frequent updates to Atlan's Chrome extension. This allows: Patching identified vulnerabilities faster through new releases while simultaneously upholding stability."}
{"id":8157,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"Mandatory code reviews specifically focused on analyzing security to help with identifying issues before these can impact customers. Once ready, both static and dynamic scanning tools rigorously test the extension codebase for any weaknesses before distribution."}
{"id":8158,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"Atlan is committed to transparency. If any post-deployment points of concern arise, Atlan will notify impacted customers promptly and address their concerns responsibly."}
{"id":8159,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"By incorporating security into each phase - secure architecture, peer reviews, robust testing, and responsible disclosure - Atlan strives to build browser extensions with both user needs and enterprise risks top of mind. Reach out to Atlan support for any questions."}
{"id":8160,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , Atlan will read: the URL of your browser tab Document Object Model (DOM) elements such as asset title, hierarchy information, text, data-test-id attributes, and more to locate an asset in a supported source tool. This list may vary depending on the source tool."}
{"id":8161,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"If you're using Atlan's browser extension on any website , it will only read the favicon, page title, and URL of your browser tab. Atlan uses the following permissions for the browser extension to work in a supported tool: activeTab - the activeTab permission allows the browser extension to temporarily access the content of the active tab as you interact with the extension."}
{"id":8162,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"This enables Atlan to display the Atlan badge and read the URL and DOM elements to locate the asset for displaying asset metadata in the sidebar. storage - the storage permission allows Atlan to store information about the locally domains configured."}
{"id":8163,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"This enables the browser extension to remember the sites that you want to use it on, even when you close and reopen your browser. host_permissions - the host permissions allow the browser extension to work specifically with Atlan tenants, which is the host in this case."}
{"id":8164,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"For example, https:\/\/atlan.com\/* , https:\/\/atlan.dev\/*. \"content_scripts\": [ { \"matches\": [\"http:\/\/*\/*\", \"https:\/\/*\/*\"] - the content_scripts key allows Atlan to inject Atlan's content script to any website you visit."}
{"id":8165,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/concepts\/atlan-browser-extension-security","title":"Atlan browser extension security | Atlan Documentation","text":"Although this content script will be injected into all webpages, it will neither be executed nor any DOM elements captured if the webpage is not a supported tool."}
{"id":8166,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":8167,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":8168,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":8169,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8170,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8171,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8172,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8173,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8174,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8175,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8176,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8177,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8178,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8179,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8180,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8181,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8182,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8183,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8184,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8185,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8186,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8187,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8188,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8189,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8190,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8191,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8192,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8193,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8194,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8195,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8196,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8197,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8198,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8199,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8200,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8201,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8202,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8203,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8204,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8205,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8206,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8207,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8208,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8209,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8210,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8211,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8212,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8213,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8214,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8215,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8216,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8217,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8218,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#access-and-enrich-context-in-flow","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8219,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":8220,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":8221,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":8222,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8223,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8224,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8225,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8226,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8227,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8228,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8229,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8230,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8231,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8232,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8233,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8234,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8235,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8236,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8237,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8238,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8239,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8240,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8241,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8242,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8243,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8244,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8245,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8246,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8247,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8248,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8249,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8250,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8251,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8252,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8253,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8254,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8255,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8256,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8257,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8258,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8259,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8260,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8261,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8262,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8263,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8264,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8265,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8266,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8267,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8268,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8269,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8270,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8271,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#search-for-metadata","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8272,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"The Atlan browser extension provides metadata context directly in your supported data tools. You can use the extension in the following Chromium-based browsers: Google Chrome and Microsoft Edge."}
{"id":8273,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Install the extension To install Atlan's browser extension: You can either: Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile. Click the four dots icon in the resulting dialog to get to integrations."}
{"id":8274,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , for Browser extension , click Install. Find the extension in the Chrome Web Store: https:\/\/chrome.google.com\/webstore\/detail\/atlan\/fipjfjlalpnbejlmmpfnmlkadjgaaheg From the upper right of any screen in Atlan, navigate to your name and then click Profile."}
{"id":8275,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8276,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the four dots icon in the resulting dialog to get to integrations. Under Apps , for Browser extension , click Install."}
{"id":8277,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"To install the Atlan browser extension: For Google Chrome, in the upper right of your screen, click Add to Chrome. When prompted for confirmation, click the Add extension button."}
{"id":8278,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store. For Google Chrome, in the upper right of your screen, click Add to Chrome."}
{"id":8279,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"When prompted for confirmation, click the Add extension button. For Microsoft Edge, follow the steps in Add an extension to Microsoft Edge from the Chrome Web Store."}
{"id":8280,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Currently, you can't install the browser extension on mobile devices or tablets. You can also install Atlan's browser extension at the workspace level."}
{"id":8281,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"To set this up, you need to be an administrator or have access to the admin console of your organization's Google account. If your organization uses managed browsers, you can configure the extension for managed browsers."}
{"id":8282,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension Once installed, configure the Atlan browser extension to get started. Optionally, Atlan admins can preconfigure custom domains for data sources , if any."}
{"id":8283,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Configure the extension as a user To configure the browser extension, once installed: If you are logged into your Atlan instance, skip to the next step. If you haven't logged into Atlan, log in to your Atlan instance when prompted."}
{"id":8284,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the Options page, to enter the URL of your Atlan instance: If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8285,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8286,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your organization uses an Atlan domain (for example, _mycompany_.atlan.com ), the Atlan instance URL appears preselected. Click Get started."}
{"id":8287,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Switch to a different Atlan domain, if required. If your organization uses a custom domain (for example, _atlan_.mycompany.com ), enter the URL of your Atlan instance and then click Get started."}
{"id":8288,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"After a successful login, the message Updated successfully appears. (Optional) If your data tools are hosted on custom domains, rather than the standard SaaS domain of each tool: Click the Configure custom domain link at the bottom."}
{"id":8289,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the dropdown on the left, select your data tool. In the text box on the right, enter the custom domain you use for that tool."}
{"id":8290,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Repeat these steps for each tool hosted on a custom domain. Click the Save button when finished."}
{"id":8291,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections. Click + Add to configure custom domains for additional data sources as required."}
{"id":8292,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Configure custom domain link at the bottom. In the dropdown on the left, select your data tool."}
{"id":8293,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8294,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. In the dropdown on the left, select your data tool."}
{"id":8295,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the text box on the right, enter the custom domain you use for that tool. Repeat these steps for each tool hosted on a custom domain."}
{"id":8296,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the Save button when finished. If your Atlan admin has preconfigured custom domains for data sources , you won't be able to update or remove these selections."}
{"id":8297,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click + Add to configure custom domains for additional data sources as required. You can now close the Options tab."}
{"id":8298,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"The extension is now ready to use! ð (Optional) Configure custom domains as an admin You need to be an admin user in Atlan to configure custom domains for data sources from the admin center."}
{"id":8299,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"To configure custom domains, from within Atlan: From the left menu of any screen, click Admin. Under Workspace , click Integrations."}
{"id":8300,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Under Apps , expand the Browser extension tile. In the Browser extension tile, for Set up your custom data source."}
{"id":8301,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":", if your data tools are hosted on custom domains rather than the standard SaaS domain of each tool, click the Configure link to configure them for users in your organization. For Connector , select a supported tool for the browser extension."}
{"id":8302,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the adjacent field, enter the URL of the custom domain for your data source. (Optional) Click + Add to add more."}
{"id":8303,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click Save to save your configuration. info ð ª Did you know?"}
{"id":8304,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"For any supported tools that you have configured, your users won't be able to update or remove these selections. They can, however, add additional custom domains for data sources."}
{"id":8305,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"For Connector , select a supported tool for the browser extension. In the adjacent field, enter the URL of the custom domain for your data source."}
{"id":8306,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Click + Add to add more. Click Save to save your configuration."}
{"id":8307,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð ª Did you know? For any supported tools that you have configured, your users won't be able to update or remove these selections."}
{"id":8308,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"They can, however, add additional custom domains for data sources. (Optional) For Download Atlan extension or share with your team , you can either install the Atlan browser extension for your own use or share the link with your users."}
{"id":8309,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Anyone with access to Atlan any admin, member, or guest user and a supported tool can use the browser extension. First, log into Atlan."}
{"id":8310,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"When using Atlan's browser extension in a supported tool , the extension only reads the URL of your browser tab no other data is accessed. If using Atlan's browser extension on any website , it only reads the favicon, page title, and URL of your browser tab."}
{"id":8311,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Learn more about Atlan browser extension security. Access and enrich context in-flow To access context for an asset, from within a supported tool: Log into the supported tool."}
{"id":8312,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Open any supported asset. In the lower-right corner of the page, click the small Atlan icon."}
{"id":8313,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"danger The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8314,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"The icon to activate Atlan is not the extension icon that appears at the top of your Chrome browser. This small Atlan icon in the lower right corner of the page is the only way to access the metadata for the asset you are viewing in another tool."}
{"id":8315,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the sidebar that appears: Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8316,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Click the tabs and links to view all context about the asset. Make changes to any of the metadata you'd like."}
{"id":8317,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Now you can understand and enrich assets without leaving your data tools themselves! ð The Atlan sidebar automatically reloads as you browse your assets in a supported tool to show details about the asset you're currently viewing."}
{"id":8318,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Your permissions in Atlan control what metadata you can see and change in the extension. The extension opens a new browser tab on Atlan's discovery page, with the results for that text!"}
{"id":8319,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð Add a resource You can link any web page as a resource to your assets in Atlan using the browser extension. To add a web page as a resource to an asset: In the top right of the web page you're viewing, click the Atlan Chrome extension."}
{"id":8320,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"In the resource clipper menu, under Link this page to an asset , select the asset to which you'd like to add the web page as a resource. Click Save to confirm your selection."}
{"id":8321,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"(Optional) Once the resource has been linked successfully, click the Open in Atlan button to view the linked asset directly in Atlan. You can now add resources to your assets in Atlan from any website!"}
{"id":8322,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"ð The Tableau extension offers native embeddings directly in your dashboards. See Enable embedded metadata in Tableau for more information."}
{"id":8323,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"Supported tools Currently, the Atlan browser extension supports assets in the following tools: Amazon QuickSight : analyses, dashboards, and datasets Databricks : databases, schemas, views, and tables dbt Cloud : models and sources in the model editor and dbt docs Google BigQuery : datasets, schemas, views, and tables IBM Cognos Analytics : folders, dashboards, packages, explorations, reports, files, data sources, and modules Looker : dashboards, explores, and folders Microsoft Power BI : dashboards, reports, dataflows, and datasets Mode : collections, reports, queries, and charts Qlik Sense Cloud : apps, datasets, sheets, and spaces Redash : queries, dashboards, and visualizations Salesforce : objects Sigma : datasets, pages, and data elements Snowflake (via Snowsight schema explorer): databases, schemas, tables, views, dynamic tables, streams, and pipes Tableau : dashboards, data sources, workbooks, and metrics. Additionally, you can choose to switch the Tableau extension to offer native embeddings directly in your dashboards."}
{"id":8324,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/use-the-atlan-browser-extension#add-a-resource","title":"Use the Atlan browser extension | Atlan Documentation","text":"See Enable embedded metadata in Tableau for more information. ThoughtSpot : liveboards, answers, visualizations, and tables MicroStrategy : dossiers, reports, documents Install the extension Configure the extension"}
{"id":8325,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"Need to redirect users to important information that's outside Atlan? You can add links to internal or external URLs wit hin an asset profile."}
{"id":8326,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"These can help your team better understand the contextual information for your asset. To add resources to your assets, follow these steps: On the Atlan homepage, click Assets in the left menu."}
{"id":8327,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"Click on an asset to open the asset profile. In the navigation bar to the right of the asset profile, click Resources."}
{"id":8328,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"Click +Add Resource and paste your URL. For Title , type a title for your resource."}
{"id":8329,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"Your resource is now ready to use! ð Once you've added a resource, click +Add in the Resource menu to add more resources for your asset."}
{"id":8330,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-a-resource","title":"Add a resource | Atlan Documentation","text":"You can also edit or delete them to curate your list of resources. You can also add any web page as a resource to your assets using Atlan's Chrome extension ."}
{"id":8331,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"Atlan metadata layers Atlan context directly onto the source application, rather than in a sidebar. This embedded experience provides users with immediate access to data lineage, quality metrics, and governance information without leaving their Tableau workflow."}
{"id":8332,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"Before enabling embedded metadata in Tableau: You must have the Atlan browser extension installed. If not, see the How to use the Atlan browser extension guide for instructions."}
{"id":8333,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"Tableau dashboards must be available through the browser Permissions required To enable this feature, you need: Admin role in Atlan Access to the Labs settings Enable embedded metadata in Tableau To enable the embedded Atlan experience in Tableau for your users: From the left menu in Atlan, click Admin. From the left menu in Atlan, click Admin."}
{"id":8334,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"Under Workspace , click Labs. Under Workspace , click Labs."}
{"id":8335,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"Under the Access Control heading of the Labs page, turn on View embedded metadata in Tableau Under the Access Control heading of the Labs page, turn on View embedded metadata in Tableau Once enabled, users with the extension installed can view Atlan metadata in Tableau dashboards. Once enabled, users with the extension installed can view Atlan metadata in Tableau dashboards."}
{"id":8336,"url":"https:\/\/docs.atlan.com\/product\/integrations\/automation\/browser-extension\/how-tos\/enable-embedded-metadata-in-tableau","title":"Enable embedded metadata in Tableau | Atlan Documentation","text":"See also Troubleshoot the Atlan browser extension Use the Atlan browser extension Tableau connector documentation Enable embedded metadata in Tableau"}
{"id":8337,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/amazon-quicksight\/references\/what-does-atlan-crawl-from-amazon-quicksight","title":"What does Atlan crawl from Amazon QuickSight? | Atlan Documentation","text":"Atlan currently supports lineage for the Amazon QuickSight connector to the following data sources: Microsoft SQL Server Atlan crawls and maps the following assets and properties from Amazon QuickSight. Atlan maps analyses from Amazon QuickSight to its QuickSightAnalysis asset type."}
{"id":8338,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/amazon-quicksight\/references\/what-does-atlan-crawl-from-amazon-quicksight","title":"What does Atlan crawl from Amazon QuickSight? | Atlan Documentation","text":"Atlan maps dashboards from Amazon QuickSight to its QuickSightDashboard asset type. Atlan maps datasets from Amazon QuickSight to its QuickSightDataset asset type."}
{"id":8339,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/amazon-quicksight\/references\/what-does-atlan-crawl-from-amazon-quicksight","title":"What does Atlan crawl from Amazon QuickSight? | Atlan Documentation","text":"Atlan maps folders from Amazon QuickSight to its QuickSightFolder asset type. Dataset fields Atlan maps dataset fields from Amazon QuickSight to its QuickSightDatasetField asset type."}
{"id":8340,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/amazon-quicksight\/references\/what-does-atlan-crawl-from-amazon-quicksight","title":"What does Atlan crawl from Amazon QuickSight? | Atlan Documentation","text":"Analysis visuals Atlan maps analysis visuals from Amazon QuickSight to its QuickSightAnalysisVisual asset type. Dashboard visuals Atlan maps dashboard visuals from Amazon QuickSight to its QuickSightDashboardVisual asset type."}
{"id":8341,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/references\/what-does-atlan-crawl-from-databricks","title":"What does Atlan crawl from Databricks? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Databricks. The following properties aren't crawled by the System tables extraction method: Table properties : partitionList , partitionCount Column properties : maxLength , precision Atlan maps databases from Databricks to its Database asset type."}
{"id":8342,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/references\/what-does-atlan-crawl-from-databricks","title":"What does Atlan crawl from Databricks? | Atlan Documentation","text":"Atlan maps schemas from Databricks to its Schema asset type. Atlan maps tables from Databricks to its Table asset type."}
{"id":8343,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/references\/what-does-atlan-crawl-from-databricks","title":"What does Atlan crawl from Databricks? | Atlan Documentation","text":"Atlan maps views from Databricks to its View asset type. Materialized views Atlan maps materialized views from Databricks to its MaterialisedView asset type."}
{"id":8344,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/references\/what-does-atlan-crawl-from-databricks","title":"What does Atlan crawl from Databricks? | Atlan Documentation","text":"To help you work seamlessly with STRUCT data types, Atlan supports nested columns up to level 30 in Databricks. You can view these columns in the Tree view or the asset sidebar of your table assets, and also explore child columns of STRUCTs nested within MAPs or ARRAYs."}
{"id":8345,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/databricks\/references\/what-does-atlan-crawl-from-databricks","title":"What does Atlan crawl from Databricks? | Atlan Documentation","text":"However, lineage for nested columns isn't supported. Atlan maps columns from Databricks to its Column asset type."}
{"id":8346,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from dbt Cloud. Atlan also supports lineage between the following: SQL tables and views materialized by dbt models, dbt seeds, dbt sources Column-level lineage for these entities Atlan only crawls dbt assets that are in the applied (built) state in dbt Cloud."}
{"id":8347,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Models must be part of a successful run to be picked up during crawling; models that are only defined in your project files but haven t been executed won t be included. For more information about project state, see Project states in dbt Cloud."}
{"id":8348,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Once you've crawled dbt , you can use dbt-specific filters for quick asset discovery: Test status - filter dbt tests that passed, failed, or have a warning or error Alias - filter by the name of a dbt model's identifier in the dbt project Unique id - filter by the unique node identifier of a dbt model Project name - filter by dbt project name, only supported for dbt Core version 1.6+ Environment name - filter by dbt environment name Job status - filter by dbt job status Last job run - filter by the last run of the dbt job Atlan's dbt connectivity also populates custom metadata to further enrich the assets in Atlan. The Atlan dbt-specific property column in the tables below gives the name of the mapped custom metadata property in Atlan."}
{"id":8349,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Atlan enables you to sync your dbt tags and update your dbt assets with the synced tags. It's also possible to map other metadata on Atlan's assets through your dbt models."}
{"id":8350,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Atlan maps tables from dbt Cloud to its Table asset type. Atlan maps columns from dbt Cloud to its Column asset type."}
{"id":8351,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Atlan maps models from dbt Cloud to its Model asset type. Atlan maps sources from dbt Cloud to its DbtSource asset type."}
{"id":8352,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/etl-tools\/dbt\/references\/what-does-atlan-crawl-from-dbt-cloud","title":"What does Atlan crawl from dbt Cloud? | Atlan Documentation","text":"Atlan maps tests from dbt Cloud to its Test asset type. Atlan maps models from dbt Core to its Seed asset type."}
{"id":8353,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Once you have crawled Google BigQuery , you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets: Tables - BigQuery labels and Is sharded filters Atlan doesn't run any table scans."}
{"id":8354,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Atlan leverages the table preview options from Google BigQuery that enable you to view data for free and without affecting any quotas using the tabledata.list API. Hence, table asset previews in Atlan are already cost-optimized."}
{"id":8355,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"However, this doesn't apply to views and materialized views. For Google BigQuery views and materialized views , Atlan sends you a cost nudge before viewing a sample data preview."}
{"id":8356,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"This informs you about the precise bytes that are spent during the execution of the query, helping you decide if you still want to run the preview. You also receive a cost nudge before querying your Google BigQuery assets."}
{"id":8357,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Google BigQuery. Atlan maps projects from Google BigQuery to its Database asset type."}
{"id":8358,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Atlan maps datasets from Google BigQuery to its Schema asset type. Table asset previews are already cost-optimized."}
{"id":8359,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Google BigQuery enables you to use the table preview options to view data for free and without affecting any quotas. Note that this isn't currently supported for Google BigQuery views and materialized views in Atlan."}
{"id":8360,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Atlan maps tables from Google BigQuery to its Table asset type. Atlan maps views from Google BigQuery to its View asset type."}
{"id":8361,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Materialized views Atlan maps materialized views from Google BigQuery to its MaterialisedView asset type. Atlan supports nested columns up to level 1 for Google BigQuery to help you enrich your semi-structured data types."}
{"id":8362,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"You can view nested columns in the asset sidebar for your table assets. Atlan maps columns from Google BigQuery to its Column asset type."}
{"id":8363,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/google-bigquery\/references\/what-does-atlan-crawl-from-google-bigquery","title":"What does Atlan crawl from Google BigQuery? | Atlan Documentation","text":"Atlan doesn't crawl primary key (PK) and foreign key (FK) information from Google BigQuery. Stored procedures Atlan maps stored procedures from Google BigQuery to its Procedure asset type."}
{"id":8364,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from IBM Cognos Analytics. Atlan also supports lineage: For packages, files, reports, and modules."}
{"id":8365,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"To upstream sources - Microsoft SQL Server and Snowflake. Field-level lineage is currently not supported."}
{"id":8366,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Atlan generates the sourceURL property for IBM Cognos Analytics assets using a combination of the host, port, and id of the asset. This allows Atlan to help you view your assets directly in IBM Cognos Analytics from the asset profile."}
{"id":8367,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Direct extraction - in addition to id , Atlan obtains the host and port values from the credentials you provided while setting up a crawler workflow. Offline extraction - in addition to id , Atlan obtains the host and port values from the parameters with which the offline extractor is executed."}
{"id":8368,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Assets marked with ð includes lineage and column-level lineage. Assets marked with ð display column information."}
{"id":8369,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Atlan maps folders from IBM Cognos Analytics to its CognosFolder asset type. Dashboards ð Atlan maps dashboards from IBM Cognos Analytics to its CognosDashboard asset type."}
{"id":8370,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Packages ð Atlan maps packages from IBM Cognos Analytics to its CognosPackage asset type. Explorations ð Atlan maps explorations from IBM Cognos Analytics to its CognosExploration asset type."}
{"id":8371,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Reports ð Atlan maps reports from IBM Cognos Analytics to its CognosReport asset type. Files ð Atlan maps files from IBM Cognos Analytics to its CognosFile asset type."}
{"id":8372,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Data sources Atlan maps data sources from IBM Cognos Analytics to its CognosDatasource asset type. Modules ð Atlan maps modules from IBM Cognos Analytics to its CognosModule asset type."}
{"id":8373,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"Atlan maps fields from IBM Cognos Analytics to its CognosColumns asset type. Based on the asset type, some attributes may not be extracted: cognosColumnRegularAggregate appears only for reports and datasets."}
{"id":8374,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/ibm-cognos-analytics\/references\/what-does-atlan-crawl-from-ibm-cognos-analytics","title":"What does Atlan crawl from IBM Cognos Analytics? | Atlan Documentation","text":"cognosColumnDatatype appears only for modules."}
{"id":8375,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Looker. Atlan also supports the following lineage: Asset-level lineage for views, models, looks, dashboards, tiles, and explores."}
{"id":8376,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Field-level lineage for views, explores, looks, tiles, and dashboards. Lineage between explore fields and dashboards."}
{"id":8377,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"This allows you to view all the fields used in a given dashboard and trace their upstream lineage to SQL columns. Cross-project lineage for Looker assets."}
{"id":8378,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"For example, if an explore includes a view from an imported project, Atlan will parse project manifest files to generate lineage. Looker refinements for views and explores."}
{"id":8379,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Atlan will parse project manifest files to generate lineage. Refined fields for views and explores are displayed with a Refinement label in Atlan."}
{"id":8380,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Currently Atlan only represents the assets marked with ð in lineage. Atlan maps connections from Looker to its Connection asset type."}
{"id":8381,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Atlan maps projects from Looker to its LookerProject asset type. Views ð Atlan maps views from Looker to its LookerView asset type."}
{"id":8382,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"To trace the upstream lineage of these views, Atlan currently supports SQL-based derived tables. Persistent derived tables (PDTs) and Liquid parameterized tables are currently not supported."}
{"id":8383,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"However, Atlan will always catalog the associated views. Atlan also supports view refinements."}
{"id":8384,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Atlan includes the fields from refinements in the parent view asset, and marks the fields with a Refinement label. You can hover over the label to view the file path and line number where the refinement is defined."}
{"id":8385,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Models ð Atlan maps models from Looker to its LookerModel asset type. Atlan maps folders from Looker to its LookerFolder asset type."}
{"id":8386,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Fields ð For explores Atlan maps fields for explores from Looker to its LookerField asset type. For views Atlan maps fields for views from Looker to its LookerField asset type."}
{"id":8387,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"For looks Atlan maps fields for looks from Looker to its LookerField asset type. For tiles Atlan maps fields for tiles from Looker to its LookerField asset type."}
{"id":8388,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"For dashboards Atlan maps fields for dashboards from Looker to its LookerField asset type. Looks ð Atlan maps looks from Looker to its LookerLook asset type."}
{"id":8389,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Dashboards ð Atlan maps dashboards from Looker to its LookerDashboard asset type. Tiles ð Atlan maps tiles from Looker to its LookerTile asset type."}
{"id":8390,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"Explores ð Atlan maps explores from Looker to its LookerExplore asset type. Atlan also supports explore refinements : For explores defined in the same model, Atlan includes the fields from refinements in the parent explore asset."}
{"id":8391,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"For explores with the same name that are defined in a different model, Atlan will create a new explore asset. In both cases, Atlan marks the fields with a Refinement label."}
{"id":8392,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/looker\/references\/what-does-atlan-crawl-from-looker","title":"What does Atlan crawl from Looker? | Atlan Documentation","text":"You can hover over the label to view the file path and line number where the refinement is defined."}
{"id":8393,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Microsoft Power BI. Once you've crawled Microsoft Power BI , you can use connector-specific filters for quick asset discovery."}
{"id":8394,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"The following filters are currently supported for these assets: Measures - External measure filter Currently Atlan only represents the assets marked with ð in lineage. For your Microsoft Power BI reports , Atlan also provides asset previews to help with quick discovery and give you the context you need."}
{"id":8395,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Atlan maps Apps from Microsoft Power BI to its PowerBIApp asset type. Atlan maps workspaces from Microsoft Power BI to its PowerBIWorkspace asset type."}
{"id":8396,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Dashboards ð Atlan maps dashboards from Microsoft Power BI to its PowerBIDashboard asset type. Data sources Atlan maps data sources from Microsoft Power BI to its PowerBIDatasource asset type."}
{"id":8397,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Datasets ð Atlan maps datasets from Microsoft Power BI to its PowerBIDataset asset type. Dataflows ð Atlan maps dataflows from Microsoft Power BI to its PowerBIDataflow asset type."}
{"id":8398,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Atlan currently only supports dataflow lineage for the following SQL sources: Microsoft SQL Server Dataflow entity columns ð Atlan maps attributes of dataflow entities from Microsoft Power BI to its PowerBIDataflowEntityColumn asset type. Reports ð Atlan maps reports from Microsoft Power BI to its PowerBIReport asset type."}
{"id":8399,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Pages ð Atlan maps pages from Microsoft Power BI to its PowerBIPage asset type. Tiles ð Atlan maps tiles from Microsoft Power BI to its PowerBITile asset type."}
{"id":8400,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Tables ð Atlan maps tables from Microsoft Power BI to its PowerBITable asset type. Columns ð Atlan maps columns from Microsoft Power BI to its PowerBIColumn asset type."}
{"id":8401,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Measures ð Atlan maps measures from Microsoft Power BI to its PowerBIMeasure asset type. Atlan supports PowerBI Measures for downstream lineage to a PowerBI Page."}
{"id":8402,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microsoft-power-bi\/references\/what-does-atlan-crawl-from-microsoft-power-bi","title":"What does Atlan crawl from Microsoft Power BI? | Atlan Documentation","text":"Dataflow entity columns ð"}
{"id":8403,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/mode\/references\/what-does-atlan-crawl-from-mode","title":"What does Atlan crawl from Mode? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Mode. Atlan maps workspaces from Mode to its ModeWorkspace asset type."}
{"id":8404,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/mode\/references\/what-does-atlan-crawl-from-mode","title":"What does Atlan crawl from Mode? | Atlan Documentation","text":"Atlan maps collections from Mode to its ModeCollection asset type. Atlan maps reports from Mode to its ModeReport asset type."}
{"id":8405,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/mode\/references\/what-does-atlan-crawl-from-mode","title":"What does Atlan crawl from Mode? | Atlan Documentation","text":"Atlan maps queries from Mode to its ModeQuery asset type. Atlan maps charts from Mode to its ModeChart asset type."}
{"id":8406,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Qlik Sense Cloud. Once you've crawled Qlik Sense Cloud , you can use connector-specific filters for quick asset discovery."}
{"id":8407,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"The following filters are currently supported for these assets: Atlan only supports asset-level lineage for the following asset types: Datasets --> Charts --> Sheets --> Apps Atlan maps spaces from Qlik Sense Cloud to its QlikSpace asset type. Atlan maps apps from Qlik Sense Cloud to its QlikApp asset type."}
{"id":8408,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"Only the app resource type is retrieved. Other types, such as qvapp or qlikview , are not crawled."}
{"id":8409,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"Atlan maps sheets from Qlik Sense Cloud to its QlikSheet asset type. Atlan maps charts from Qlik Sense Cloud to the QlikChart asset type and catalogs only those linked to dataset fields."}
{"id":8410,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"For example, table charts are crawled because their columns represent dataset dimensions or measures. UI elements that do not reference dataset fields - such as filters, buttons, and text elements - are ignored."}
{"id":8411,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"These elements are not considered charts and are not crawled: filterpane , qlik-button-for-navigation , VizlibAdvancedTextObject , listbox , action-button , VizlibFilter , variable , text-image , VizlibLineObject. Atlan maps datasets from Qlik Sense Cloud to the QlikDataset asset type."}
{"id":8412,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/qlik-sense-cloud\/references\/what-does-atlan-crawl-from-qlik-sense-cloud","title":"What does Atlan crawl from Qlik Sense Cloud? | Atlan Documentation","text":"Datasets loaded through the Data Load Editor are called implicit datasets in Atlan and appear under this type."}
{"id":8413,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/redash\/references\/what-does-atlan-crawl-from-redash","title":"What does Atlan crawl from Redash? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Redash. Once you've crawled Redash , you can use connector-specific filters for quick asset discovery."}
{"id":8414,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/redash\/references\/what-does-atlan-crawl-from-redash","title":"What does Atlan crawl from Redash? | Atlan Documentation","text":"The following filters are currently supported for these assets: Queries - Schedule, Is Published, and Redash tags filters Visualizations - Type filter Dashboards - Redash tags filter Currently, Atlan only represents the assets marked with ð in lineage. Queries ð Atlan maps queries from Redash to its RedashQuery asset type."}
{"id":8415,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/redash\/references\/what-does-atlan-crawl-from-redash","title":"What does Atlan crawl from Redash? | Atlan Documentation","text":"Dashboards ð Atlan maps dashboards from Redash to its RedashDashboard asset type. Visualizations ð Atlan maps visualization elements from Redash to its RedashVisualization asset type."}
{"id":8416,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/crm\/salesforce\/references\/what-does-atlan-crawl-from-salesforce","title":"What does Atlan crawl from Salesforce? | Atlan Documentation","text":"Atlan only performs GET requests on these five endpoints: sObject Basic Information Each endpoint will be set in its own OAuth client session. For every API request, it will hit the Salesforce login endpoint, which means there will be at least five (same as the number of endpoints above) login entries in your Salesforce account's login history within the duration of the scheduled workflow run."}
{"id":8417,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/crm\/salesforce\/references\/what-does-atlan-crawl-from-salesforce","title":"What does Atlan crawl from Salesforce? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Salesforce. Once you've crawled Salesforce , you can use connector-specific filters for quick asset discovery."}
{"id":8418,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/crm\/salesforce\/references\/what-does-atlan-crawl-from-salesforce","title":"What does Atlan crawl from Salesforce? | Atlan Documentation","text":"The following filters are currently supported for these assets: Fields - Is encrypted and Is required filters Atlan maps organizations from Salesforce to its SalesforceOrganization asset type. Atlan maps objects from Salesforce to its SalesforceObject asset type."}
{"id":8419,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/crm\/salesforce\/references\/what-does-atlan-crawl-from-salesforce","title":"What does Atlan crawl from Salesforce? | Atlan Documentation","text":"Atlan maps fields from Salesforce to its SalesforceField asset type. Atlan maps reports from Salesforce to its SalesforceReport asset type."}
{"id":8420,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/crm\/salesforce\/references\/what-does-atlan-crawl-from-salesforce","title":"What does Atlan crawl from Salesforce? | Atlan Documentation","text":"Atlan maps dashboards from Salesforce to its SalesforceDashboard asset type."}
{"id":8421,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/sigma\/references\/what-does-atlan-crawl-from-sigma","title":"What does Atlan crawl from Sigma? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Sigma. Currently, Atlan only represents the assets marked with ð in lineage."}
{"id":8422,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/sigma\/references\/what-does-atlan-crawl-from-sigma","title":"What does Atlan crawl from Sigma? | Atlan Documentation","text":"For your Sigma workbooks , Atlan also provides asset previews to help with quick discovery and give you the context you need. Workbooks ð Atlan maps workbooks from Sigma to its SigmaWorkbook asset type."}
{"id":8423,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/sigma\/references\/what-does-atlan-crawl-from-sigma","title":"What does Atlan crawl from Sigma? | Atlan Documentation","text":"Pages ð Atlan maps pages from Sigma to its SigmaPage asset type. Data elements ð Atlan maps table, pivot table, and visualization elements from Sigma to its SigmaDataElement asset type."}
{"id":8424,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/sigma\/references\/what-does-atlan-crawl-from-sigma","title":"What does Atlan crawl from Sigma? | Atlan Documentation","text":"Data element fields ð Atlan maps table, pivot table, and visualization element fields from Sigma to its SigmaDataElementField asset type. Atlan maps datasets from Sigma to its SigmaDataset asset type."}
{"id":8425,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/sigma\/references\/what-does-atlan-crawl-from-sigma","title":"What does Atlan crawl from Sigma? | Atlan Documentation","text":"Data elements ð Data element fields ð"}
{"id":8426,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Snowflake. Once you've crawled Snowflake , you can use connector-specific filters for quick asset discovery."}
{"id":8427,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"The following filters are currently supported for Snowflake assets: Streams - Source type and Stale filters Functions - Language, Function type, Is secure, and Is external filters Snowflake tags and tag values Atlan supports lineage for the following asset types: External Named Stages Pipe Table Internal Named Stages Pipe Table (auto-ingest not recommended) Not supported for External or Iceberg Tables Atlan maps databases from Snowflake to its Database asset type. Atlan maps schemas from Snowflake to its Schema asset type."}
{"id":8428,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Atlan maps tables from Snowflake to its Table asset type. For Iceberg tables In addition to the table metadata above, Atlan supports additional metadata for Iceberg tables crawled from Snowflake."}
{"id":8429,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Atlan currently supports fetching metadata for Iceberg tables only for the information schema extraction method. Atlan maps views from Snowflake to its View asset type."}
{"id":8430,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Materialized views Atlan maps materialized views from Snowflake to its MaterialisedView asset type. External tables Atlan maps external tables from Snowflake to its Table asset type."}
{"id":8431,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Atlan maps columns from Snowflake to its Column asset type. Atlan maps stages from Snowflake to its Stage asset type."}
{"id":8432,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Atlan maps streams from Snowflake to its Stream asset type. Atlan maps pipes from Snowflake to its Pipe asset type."}
{"id":8433,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"User-defined functions Atlan maps user-defined functions (UDFs) from Snowflake to its Function asset type. Atlan currently does not support lineage for user-defined functions from Snowflake."}
{"id":8434,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/data-warehouses\/snowflake\/references\/what-does-atlan-crawl-from-snowflake","title":"What does Atlan crawl from Snowflake? | Atlan Documentation","text":"Dynamic tables Atlan maps dynamic tables from Snowflake to its DynamicTable asset type."}
{"id":8435,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from Tableau. Once you've crawled Tableau , you can use connector-specific filters for quick asset discovery."}
{"id":8436,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"The following filters are currently supported for these assets: Projects - filter Tableau assets by projects, including nested projects Data sources - Is published filter For your Tableau worksheets and dashboards , Atlan also provides asset previews to help with quick discovery and give you the context you need. You may need to disable clickjack protection for Tableau asset previews to load."}
{"id":8437,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Did you know? Lineage to dashboards may appear incomplete or missing if worksheets are not crawled."}
{"id":8438,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Additionally, Tableau assets that haven't been refreshed since May 27, 2025, won't display the new column-level lineage (CLL) or updated lineage paths. Atlan supports lineage for the following: Asset Lineage - Datasource to Dashboard, Datasource to Worksheet, Datasource to Workbook Column Level Lineage - Supported for Datasource to Worksheet and Worksheet to Dashboard Atlan maps sites from Tableau to its TableauSite asset type."}
{"id":8439,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan maps projects from Tableau to its TableauProject asset type. Due to limitations at source, Atlan won't be able to crawl Tableau flows if you use the JWT bearer authentication method."}
{"id":8440,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan maps flows from Tableau to its TableauFlow asset type. Tableau has retired metrics methods in API 3.22 for Tableau Cloud and Tableau Server version 2024.2."}
{"id":8441,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"If you're using Tableau API version 3.22 or higher, metadata for metrics is unavailable in Atlan. Atlan maps metrics from Tableau to its TableauMetric asset type."}
{"id":8442,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan maps workbooks from Tableau to its TableauWorkbook asset type. Atlan maps worksheets from Tableau to its TableauWorksheet asset type."}
{"id":8443,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan maps dashboards from Tableau to its TableauDashboard asset type. Data sources Atlan maps data sources (embedded and published) from Tableau to its TableauDatasource asset type."}
{"id":8444,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Data source fields Atlan maps data source fields and column fields from Tableau to its TableauDatasourceField asset type. Custom SQL Atlan parses custom SQL queries used in Tableau data sources to extract lineage information."}
{"id":8445,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"This process identifies the relationships between data assets based on the SQL logic defined within Tableau. Calculated fields Atlan maps calculated fields from Tableau to its TableauCalculatedField asset type."}
{"id":8446,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/tableau\/references\/what-does-atlan-crawl-from-tableau","title":"What does Atlan crawl from Tableau? | Atlan Documentation","text":"Atlan calculates lineage for Tableau as follows: Lineage is currently not supported for Tableau flows and metrics. Data source fields"}
{"id":8447,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/thoughtspot\/references\/what-does-atlan-crawl-from-thoughtspot","title":"What does Atlan crawl from ThoughtSpot? | Atlan Documentation","text":"Once you've crawled ThoughtSpot , you can use connector-specific filters for quick asset discovery. The following filters are currently supported for all ThoughtSpot assets: Tags and chart type filters Atlan supports lineage for the following ThoughtSpot assets: Answers - upstream lineage to tables, views, or worksheets from multiple sources (if applicable), no downstream lineage Visualizations - upstream lineage to tables, views, or worksheets from multiple sources (if applicable) Liveboards - upstream lineage to visualizations Tables - upstream lineage to source tables, and column-level lineage between ThoughtSpot tables and worksheets Views - upstream lineage to ThoughtSpot tables or worksheets, and column-level lineage between ThoughtSpot views and worksheets Worksheets - upstream lineage to ThoughtSpot tables or views from multiple sources (if applicable) Atlan crawls and maps the following assets and properties from ThoughtSpot."}
{"id":8448,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/thoughtspot\/references\/what-does-atlan-crawl-from-thoughtspot","title":"What does Atlan crawl from ThoughtSpot? | Atlan Documentation","text":"Currently, Atlan only represents the assets marked with ð in lineage. Answers ð Atlan maps answers from ThoughtSpot to its ThoughtSpotAnswer asset type."}
{"id":8449,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/thoughtspot\/references\/what-does-atlan-crawl-from-thoughtspot","title":"What does Atlan crawl from ThoughtSpot? | Atlan Documentation","text":"Visualizations ð Atlan maps visualizations from ThoughtSpot to its ThoughtspotDashlet asset type. Liveboards ð Atlan maps Liveboards from ThoughtSpot to its ThoughtspotLiveboard asset type."}
{"id":8450,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/thoughtspot\/references\/what-does-atlan-crawl-from-thoughtspot","title":"What does Atlan crawl from ThoughtSpot? | Atlan Documentation","text":"Tables ð Atlan maps tables from ThoughtSpot to its ThoughtspotTable asset type. Views ð Atlan maps views from ThoughtSpot to its ThoughtspotView asset type."}
{"id":8451,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/thoughtspot\/references\/what-does-atlan-crawl-from-thoughtspot","title":"What does Atlan crawl from ThoughtSpot? | Atlan Documentation","text":"Worksheets ð Atlan maps worksheets from ThoughtSpot to its ThoughtspotWorksheet asset type. Columns ð Atlan maps columns from ThoughtSpot to its ThoughtspotColumn asset type."}
{"id":8452,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Atlan crawls and maps the following assets and properties from MicroStrategy. Currently Atlan only represents the assets marked with ð in lineage."}
{"id":8453,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Once you've crawled MicroStrategy , you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets: Projects , attributes , facts , metrics , cubes , reports , documents , dossiers , and visualizations - Is Certified filter Cubes , reports , and visualizations - Type filter Atlan maps projects from MicroStrategy to its MicroStrategyProject asset type."}
{"id":8454,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Atlan maps attributes from MicroStrategy to its MicroStrategyAttribute asset type. Atlan maps facts from MicroStrategy to its MicroStrategyFact asset type."}
{"id":8455,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Atlan maps metrics from MicroStrategy to its MicroStrategyMetric asset type. Cubes ð Atlan maps cubes from MicroStrategy to its MicroStrategyCube asset type."}
{"id":8456,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Reports ð Atlan maps reports from MicroStrategy to its MicroStrategyReport asset type. Documents ð Atlan maps documents from MicroStrategy to its MicroStrategyDocument asset type."}
{"id":8457,"url":"https:\/\/docs.atlan.com\/apps\/connectors\/business-intelligence\/microstrategy\/references\/what-does-atlan-crawl-from-microstrategy","title":"What does Atlan crawl from MicroStrategy? | Atlan Documentation","text":"Dossiers ð Atlan maps dossiers from MicroStrategy to its MicroStrategyDossier asset type. Atlan maps dossier visualizations from MicroStrategy to its MicroStrategyVisualization asset type."}
{"id":8458,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8459,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8460,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8461,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8462,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8463,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8464,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8465,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8466,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8467,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#__docusaurus_skipToContent_fallback","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8468,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8469,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8470,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8471,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8472,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8473,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8474,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8475,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8476,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8477,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#user-management","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8478,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8479,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8480,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8481,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8482,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8483,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8484,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8485,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8486,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8487,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#add-and-manage-users-from-the-admin-center","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8488,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"You will need to be an admin user in Atlan to invite users. Usernames in Atlan are of a permanent nature."}
{"id":8489,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Atlan uses usernames as a unique identifier across the platform and does not support making any changes to them. When logging into Atlan for the first time, ensure that you configure your username as per your preference."}
{"id":8490,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Without SSO To invite new users to Atlan, without SSO: From the left menu of any screen, click Admin. From the left menu of any screen, click Admin."}
{"id":8491,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Under Workspace click Users. Under Workspace click Users."}
{"id":8492,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Click the Invite Users button. Click the Invite Users button."}
{"id":8493,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Under Invite users to Default enter one or more email addresses of the users to invite. danger Atlan does not allow the use of disposable email addresses."}
{"id":8494,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"You will receive an error indicating this if you attempt to send an invitation to one. Under Invite users to Default enter one or more email addresses of the users to invite."}
{"id":8495,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Atlan does not allow the use of disposable email addresses. You will receive an error indicating this if you attempt to send an invitation to one."}
{"id":8496,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"(Optional) To the right of each email, click Member to change the role of the user. (Optional) To the right of each email, click Member to change the role of the user."}
{"id":8497,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Click the Send Invite button. Click the Send Invite button."}
{"id":8498,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Your user(s) will now receive an email with a link to sign up on Atlan! ð Note that the invitation link will remain valid for 7 days."}
{"id":8499,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"If the link expires, you can resend the invitation to your new users. With SSO When SSO is enforced , you will not be able to invite users in Atlan - they can only be invited through the SSO provider."}
{"id":8500,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Any users mapped to the SAML app can log into Atlan via SSO. A user profile will be generated for them automatically, if one does not already exist."}
{"id":8501,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/invite-new-users","title":"Invite new users | Atlan Documentation","text":"Admins can also assign default user roles for SSO to give appropriate permissions to users as soon as they log into Atlan."}
{"id":8502,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"You will need to be an admin user in Atlan to manage other users. As an admin user in Atlan, you can: Change a user's role Temporarily disable or permanently remove a user Reactivate a disabled user Change a user's role To change a user's role: From the left menu of any screen, click Admin."}
{"id":8503,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Under Workspace , click Users. Under the Role column, on a given user's row, click the role name."}
{"id":8504,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Under Change Role , select the new role to give the user, and then click Change. That's it, the user's role has now been changed!"}
{"id":8505,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"ð Deactivate a user To deactivate a user, you can either disable or remove them from Atlan. For example, you can disable a user to temporarily suspend their access to Atlan and reactivate it at a future date."}
{"id":8506,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"If a user leaves the organization, you can remove their access. Atlan recommends that you proceed with caution if you want remove a user."}
{"id":8507,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"While a disabled user can be reactivated , removing a user is a permanent and irreversible action. If a removed user needs to be restored to Atlan later on, you will need to add them as a new user."}
{"id":8508,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"You can remove users irrespective of whether you're using basic authentication, SSO, or SCIM provisioning. Note that if you're using SCIM provisioning, disabling or reactivating users in Atlan is not allowed."}
{"id":8509,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"If a SCIM-provisioned user is unassigned from the Atlan app in the identity provider, that user will be disabled in Atlan as well. Only then will you have the option to remove that user permanently."}
{"id":8510,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"To deactivate a user: From the left menu of any screen, click Admin. Under Workspace , click Users."}
{"id":8511,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Scroll the table all the way to the right, if necessary. On an active user's row, click the 3-dot menu button and then from the dropdown: To temporarily remove a user, click Disable user and then click Disable."}
{"id":8512,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"To permanently remove a user, click Remove user. In the Remove user dialog, for Transfer ownership , select an existing user to transfer any and all assets and workflows owned by the user you want to remove."}
{"id":8513,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Depending on the volume of assets to be transferred, the user removal process may take 15 minutes to a few hours to complete. You will be notified via email upon completion."}
{"id":8514,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Check the I understand this action can't be reversed and want to permanently remove this user box. Click Remove user."}
{"id":8515,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"To temporarily remove a user, click Disable user and then click Disable. To permanently remove a user, click Remove user."}
{"id":8516,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"In the Remove user dialog, for Transfer ownership , select an existing user to transfer any and all assets and workflows owned by the user you want to remove. Depending on the volume of assets to be transferred, the user removal process may take 15 minutes to a few hours to complete."}
{"id":8517,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"You will be notified via email upon completion. Check the I understand this action can't be reversed and want to permanently remove this user box."}
{"id":8518,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Click Remove user. In the Remove user dialog, for Transfer ownership , select an existing user to transfer any and all assets and workflows owned by the user you want to remove."}
{"id":8519,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Depending on the volume of assets to be transferred, the user removal process may take 15 minutes to a few hours to complete. You will be notified via email upon completion."}
{"id":8520,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Check the I understand this action can't be reversed and want to permanently remove this user box. Click Remove user."}
{"id":8521,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"That's it, the user is now deactivated and will no longer be able to access Atlan. ð ¢ Deactivated user accounts do not count towards the total number of Atlan licenses procured by your organization."}
{"id":8522,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"If the user you want to disable is a connection admin , you will need to ensure that other users can manage the connection before disabling the account. You can modify a connection to add more connection admins."}
{"id":8523,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Reactivate a user You can only reactivate a disabled user's account. To reactivate a user: From the left menu of any screen, click Admin."}
{"id":8524,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"Under Workspace , click Users. Scroll the table all the way to the right, if necessary."}
{"id":8525,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"On a deactivated user's row, click the 3-dot menu button. Click Enable user , and then Enable."}
{"id":8526,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"That's it, the user can once again access Atlan! ð Atlan has a retention policy of 60 days for user login events."}
{"id":8527,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/users-and-groups\/how-tos\/manage-users","title":"Manage users | Atlan Documentation","text":"If a user has not logged into Atlan for more than 60 days, the Last Active column will display - for the user. Change a user's role Deactivate a user Reactivate a user"}
{"id":8528,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/sso","title":"SSO Integration | Atlan Documentation","text":"Get started How to enable Azure AD for SSO How to enable Google for SSO How to enable JumpCloud for SSO How to enable Okta for SSO How to enable OneLogin for SSO How to enable SAML 2.0 for SSO Authenticate SSO credentials to query data Authenticate SSO credentials to view sample data How to limit SSO automatically creating users when they log in How to set default user roles for SSO SSO integration with PingFederate using SAML Troubleshooting connector-specific SSO authentication Why do I get a 404 error when using PingFederate SSO? Why do I get an authentication error when logging in via Okta for the first time?"}
{"id":8529,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/sso","title":"SSO Integration | Atlan Documentation","text":"Why do I get an error while logging in via Google dashboard? Unable to log into Atlan via SSO due to an internal error from Microsoft Defender Can Atlan integrate with multiple Azure AD tenants within a single instance?"}
{"id":8530,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/sso","title":"SSO Integration | Atlan Documentation","text":"Can we use a Microsoft SSO login? What type of user provisioning does Atlan support for SSO integrations?"}
{"id":8531,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/sso","title":"SSO Integration | Atlan Documentation","text":"When does Atlan become a personal data processor or subprocessor? Why did my users not receive an invite email from Atlan?"}
{"id":8532,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/scim\/how-tos\/configure-scim-provisioning","title":"Configure SCIM provisioning | Atlan Documentation","text":"You can automate the process of provisioning and deprovisioning your users and groups in Atlan with System for Cross-domain Identity Management (SCIM). Atlan supports SCIM 2.0 for SCIM provisioning."}
{"id":8533,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/scim\/how-tos\/configure-scim-provisioning","title":"Configure SCIM provisioning | Atlan Documentation","text":"SCIM provisioning works in combination with your single sign-on (SSO) setup. Setting up SCIM enables you to manage all your users from one central location."}
{"id":8534,"url":"https:\/\/docs.atlan.com\/product\/integrations\/identity-management\/scim\/how-tos\/configure-scim-provisioning","title":"Configure SCIM provisioning | Atlan Documentation","text":"Atlan currently supports SCIM provisioning for the following SSO providers: For more questions about SCIM provisioning, head over to Troubleshooting SCIM provisioning ."}
{"id":8535,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"You will need to be an admin user in Atlan to configure SMTP. The default SMTP setup is preconfigured by Atlan."}
{"id":8536,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"You should only update this configuration if you want to set up custom SMTP. Atlan uses SMTP to send emails, primarily for things like inviting users, login failure alerts, and scheduled queries."}
{"id":8537,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"We provide an embedded SMTP server to do this, out-of-the-box. If desired, you can override this embedded SMTP se rver with your own."}
{"id":8538,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"To override the embedded SMTP server: From the left menu of any screen, click Admin. Under Workspace , click SMTP."}
{"id":8539,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"Fill in the configuration of your SMTP server, at least: For Host the fully-qualified hostname of the SMTP server. For From Email the email address that should be used to send emails from the server."}
{"id":8540,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"For Username the username required by your SMTP server. For Host the fully-qualified hostname of the SMTP server."}
{"id":8541,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"For From Email the email address that should be used to send emails from the server. For Username the username required by your SMTP server."}
{"id":8542,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"At the bottom of the page, click the Test SMTP Config button to test your configuration. This will attempt to send an email to your profile's email address."}
{"id":8543,"url":"https:\/\/docs.atlan.com\/product\/integrations\/communication\/smtp-and-announcements\/how-tos\/configure-smtp","title":"Configure SMTP | Atlan Documentation","text":"Once you successfully receive the test email, at the bottom of the page click Save ."}
{"id":8544,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8545,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8546,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8547,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8548,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8549,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8550,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8551,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8552,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8553,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#manage-access-control-from-the-governance-center","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8554,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"You can customize access for users through several mechanisms. User roles The most general mechanism is a user role."}
{"id":8555,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"These define the very broad permissions a user has in Atlan - for example, whether they can administer other users, or only discover metadata. When it comes to what metadata and data a user can access, though, we need to use the additional mechanisms below."}
{"id":8556,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Connection admins Connection admins are users who manage connectivity to a data source. By default, these users can: Read and write all metadata on assets from that connection."}
{"id":8557,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Preview and query the data in all data assets from that connection. Manage access policies to grant others access to the assets from that connection."}
{"id":8558,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"You define the connection admin when crawling a new data source for the first time. A connection admin can also extend the list of connection admins on their connection at any time."}
{"id":8559,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Access policies A user must be both an admin user and a connection admin to define access policies for the connection's assets. Access policies either allow or restrict access to certain assets."}
{"id":8560,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"These allow you to be much more creative (and granular) about access than the all-or-nothing privileges of connection admins. You start by defining which assets to control with each policy."}
{"id":8561,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"There are two complementary mechanisms to do this in Atlan - personas and purposes. Once you have defined the subset of assets, you can then define granular access to both metadata and data: Metadata policies Metadata policies control what users can do with the assets' metadata."}
{"id":8562,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Through them, you can control who can: Read : view an asset's activity log, custom metadata, and SQL queries Update : change asset metadata, including description, certification, owners, README, and resources Update Custom Metadata Values for the assets Add Tags to the assets Remove Tags from the assets Create : create new assets within the selected connection (via API) Delete : delete assets within the selected connection (via API) Data policies Data policies control what users can do with the assets' data. Through them, you can control who can: Query and preview the data within the assets Whether to hide any data, through various masking techniques: Show first 4 : replaces all the data with X except the first 4 characters of data."}
{"id":8563,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"For example 1234 5678 9012 3456 would become 1234XXXX. Show last 4 : replaces all the data with X except the last 4 characters of data."}
{"id":8564,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"For example 1234 5678 9012 3456 would become XXXX3456. Hash : replaces the data with a consistent hashed value."}
{"id":8565,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Because the hash is consistent you can still join on it across assets. For example 1234 5678 9012 3456 would become f43jknscakc12nk21ak."}
{"id":8566,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Nullify : replaces the data with the null value. For example 1234 5678 9012 3456 would become null."}
{"id":8567,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Redact : replaces all alphabetic data with x and all numeric data with 0. For example 1234 Street Name would become 0000 Xxxxxx Xxxx."}
{"id":8568,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Show first 4 : replaces all the data with X except the first 4 characters of data. For example 1234 5678 9012 3456 would become 1234XXXX."}
{"id":8569,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Show last 4 : replaces all the data with X except the last 4 characters of data. For example 1234 5678 9012 3456 would become XXXX3456."}
{"id":8570,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Hash : replaces the data with a consistent hashed value. Because the hash is consistent you can still join on it across assets."}
{"id":8571,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"For example 1234 5678 9012 3456 would become f43jknscakc12nk21ak. Nullify : replaces the data with the null value."}
{"id":8572,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"For example 1234 5678 9012 3456 would become null. Redact : replaces all alphabetic data with x and all numeric data with 0."}
{"id":8573,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"For example 1234 Street Name would become 0000 Xxxxxx Xxxx. Glossary policies Glossary policies can only be defined through personas."}
{"id":8574,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"All the mechanisms above can coexist. This is powerful, but can also be a bit overwhelming to think about."}
{"id":8575,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"What takes priority when a user is under the control of all these mechanisms? ð µ ð « It's actually not as bad as you might think - only these three rules: Access is denied by default (implicitly) By default, users will not have the permissions listed above."}
{"id":8576,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"This remains true until you explicitly grant a user a permission. For example, imagine you have not set up any access policies and a new user joins."}
{"id":8577,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"They will not have any of the permissions above against any assets in Atlan. Explicit grants (allows) are combined When you grant a user a permission, this is combined with all other permissions you have granted the user."}
{"id":8578,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Continuing our example, imagine you add the new user to a group defined as the connection admins for Snowflake. The user will now have full read\/write access to all metadata for Snowflake assets, and be able to query and preview the data in those assets."}
{"id":8579,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Then you add the user to a persona that gives read\/write access to a Looker project. The user will now have access to all Snowflake assets and a Looker project's assets."}
{"id":8580,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"Explicit restrictions (denies) take priority When you explicitly deny a user a permission, this takes priority over all other permissions you have granted the user. Continuing our example, imagine you define a purpose with a data policy that masks PII data."}
{"id":8581,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"The user will still have full read\/write access to all metadata for Snowflake assets and a Looker project's assets. In general, they will still be able to query and preview the data in the Snowflake assets."}
{"id":8582,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"However, any PII data in Snowflake will now be masked. Then you add a metadata policy to the purpose that denies permission to remove the PII tag."}
{"id":8583,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"The user will no longer have full read\/write access to all metadata for Snowflake assets and a Looker project's assets. The user can no longer remove the PII tag from any of these assets."}
{"id":8584,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/custom-metadata\/how-tos\/control-access-metadata-data","title":"Control access to metadata and data? | Atlan Documentation","text":"The combination of mechanisms in the example above shows their power. Through a small number of controls we can define wide-ranging but granular access permissions."}
{"id":8585,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"Personas define policies to control which users can (or cannot) take certain actions on specific assets. They address a combination of two main objectives: Curating the assets that are relevant to a team of users Controlling the actions users can take on those assets (querying data, updating metadata, etc) Think of personas as a way of curating assets for a group of users."}
{"id":8586,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"Team-based personalization Personas curate the assets that are discoverable by users, and the detailed metadata shown. In this way, users can focus on only those assets (and metadata) relevant to their role in the organization."}
{"id":8587,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"This reduces distractions and \"noise\" for users. For example, a team of marketers may only work with a few data sets and dashboards."}
{"id":8588,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"Rather than flooding the marketers with details about every asset in your company, you want to focus their attention. Broad-brush access control Combined with the personalization are broad-brush access control policies."}
{"id":8589,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"When defining the policies in a persona, you not only select the assets but also what actions users can take on those assets. For example, your team of marketers should be able to describe their dashboards."}
{"id":8590,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"But perhaps they should only be able to see the shared data sets that feed those dashboards, not change their descriptions or tags. You can define the persona so that they can read and write the dashboards, but only read the data sets."}
{"id":8591,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-personas","title":"What are personas? | Atlan Documentation","text":"Broad-brush access control"}
{"id":8592,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Purposes provide ways to interact with tagged assets. They address two main objectives: Grouping assets together in ways they may be used by many teams - for example, by project or domain Controlling access to very granular, typically sensitive data When defining a purpose , you choose its tags."}
{"id":8593,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Atlan then considers all assets with at least one of those tags as part of the purpose. Think of purposes as a way of further protecting particularly sensitive data."}
{"id":8594,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Even if a user can see data in a table, you may not want them to see one or two sensitive columns within that table. Asset curation by domain One way you can use purposes is to curate assets."}
{"id":8595,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"In this approach, the purpose's tag tends to be a domain. For example, this could be a project or an area of your organization's business."}
{"id":8596,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Through the purpose, you can grant permissions to assets with that tag. With purposes, any future assets given a tag will gain the same permissions - no policy changes needed."}
{"id":8597,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Granular data protection The other way you can use purposes is to enforce granular data protection. In this approach, the purpose's tag tends to be some level of information sensitivity."}
{"id":8598,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"For example, this could be personally-identifiable information (PII) or confidential internal financial metrics. These sensitivity tags will tend to be against granular data assets - often columns."}
{"id":8599,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Personas tend to control permissions at a broader level, for example entire data sources, databases or schemas. Through these more granular tags, purposes give you more fine-grained control."}
{"id":8600,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"And you can layer this on top of the permissions granted by personas. For example, you might grant permission to preview and query a database to a group of users through a persona."}
{"id":8601,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"But you don't want those users to be able to see any PII data - specific columns - wherever they appear in the database. There could be hundreds of these columns, scattered across thousands of tables."}
{"id":8602,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"By tagging the columns, you can restrict access to them through a single policy in a purpose. This way you don't need to maintain many separate per-column policies through a persona."}
{"id":8603,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/access-control\/concepts\/what-are-purposes","title":"What are purposes? | Atlan Documentation","text":"Asset curation by domain Granular data protection"}
{"id":8604,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You must be an admin user in Atlan to enable , create , and manage governance workflows. Anyone with access to Atlan - admin, member, or guest user - can use the inbox."}
{"id":8605,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You can streamline your data governance requirements in Atlan with governance workflows and manage alerts, approvals, and tasks using the inbox. Governance workflows enable you to set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8606,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"For example, instead of allowing your users to directly query data or update the certification status of an asset, you can specify assets that require advanced controls and create governance workflows to govern them. These workflows will run in the background, ensure that all required approvals are in place, and only then approve users with appropriate permissions to perform any action."}
{"id":8607,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You can use governance workflows to ensure: Risk mitigation - determine how data is used and shared in your organization with automated access policies. Data security - manage requests for data access and processing to only allow access to authorized individuals or teams."}
{"id":8608,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Metadata change management - monitor and audit metadata changes to align with established organizational standards. New entity creation - manage and audit documentation of business context such as glossaries and tags to align with established organizational standards."}
{"id":8609,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Policy compliance - set up repeatable processes and approval flows for your data assets in Atlan to adhere to regulatory requirements - currently only applicable if you have also enabled the policy center module. Workflow properties A common set of properties are applicable to all governance workflows in Atlan: Only an admin user can create, update, or delete governance workflows."}
{"id":8610,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Out-of-the-box workflow templates. Predefined steps based on workflow selection."}
{"id":8611,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Must be associated with an asset type or action. Set up auto-approval rules for users, groups, or owners based on metadata attributes and policies."}
{"id":8612,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Activity logs for all workflows available by default. Visibility into the transition states of a workflow."}
{"id":8613,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Overlapping workflows - governance workflows provide you with the flexibility of creating workflows per team or business domain on the same set of assets instead of creating one complex workflow to cover all your use cases. Atlan will handle all the complexities, only allowing approvals to go through once all approval conditions have been met."}
{"id":8614,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Workflow templates You can choose from the following workflow templates to govern your assets and manage access: Change management This template allows you to control changes to metadata within your organization's data management and governance framework. Use cases include requests to: Add, update, and remove descriptions manually and using Atlan AI Add, update, and remove certificates Add, update, and remove an alias Add, update, and remove owners Attach, update, and remove tags Add, update, and remove custom metadata Add, update, and remove domains Add, update, and remove READMEs Add, update, and remove announcements Change management workflows will override any permissions assigned through user roles or access policies."}
{"id":8615,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"For example, even for users with edit access , metadata update requests will go through change management workflows. If there are no change management workflows in place, then users with edit access will be able to update metadata while users without edit access will only be able to suggest changes to metadata."}
{"id":8616,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"New entity creation This template allows you to control the creation and publication of new entities in Atlan. The new entity creation workflow will override existing glossary policies and user role permissions to create new entities."}
{"id":8617,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Creation of the following entities is currently supported for the new entity creation workflow: Data products : Creation of a new data product Change of a data product's status from Sunset , Archived , or Draft to Published Creation of a new data product Change of a data product's status from Sunset , Archived , or Draft to Published Access management This template allows you to automate the process of requesting, approving, and revoking access to data assets in Atlan. It includes the combination of a self-service approach as well as mandating human intervention for approval."}
{"id":8618,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You can also revoke data access in Atlan or other data sources. For data sources other than Atlan, you can configure additional actions to revoke data access in the data source."}
{"id":8619,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Use cases include requests to query data or view sample data for the following supported asset types - tables, views, and materialized views. Grant access in Atlan - allow requesters to request data access for querying data in Insights and previewing sample data within Atlan only."}
{"id":8620,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Raise Jira ticket to grant or revoke data access on source - allow requesters to request or revoke data access for any tool. Atlan will create a support ticket in Jira Cloud for your team to grant or revoke data access and display the status of your request in Atlan."}
{"id":8621,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You will need to: Integrate Jira Cloud and Atlan. Link your individual Jira Cloud account to Atlan."}
{"id":8622,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Install or register a webhook. Create an access management workflow to enable or revoke access everywhere using Jira."}
{"id":8623,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Add a Jira project and issue type and specify an issue status while creating the data access workflow. Your users will be granted access or their access will be revoked once the request is approved in Jira."}
{"id":8624,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Integrate Jira Cloud and Atlan. Link your individual Jira Cloud account to Atlan."}
{"id":8625,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Install or register a webhook. Create an access management workflow to enable or revoke access everywhere using Jira."}
{"id":8626,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Add a Jira project and issue type and specify an issue status while creating the data access workflow. Your users will be granted access or their access will be revoked once the request is approved in Jira."}
{"id":8627,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Add a Jira project and issue type and specify an issue status while creating the data access workflow. Your users will be granted access or their access will be revoked once the request is approved in Jira."}
{"id":8628,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Raise ServiceNow request to grant or revoke data access on source - allow requesters to request or revoke data access for any tool. Atlan will create a request in the Atlan Data Access catalog for your team in ServiceNow to grant or revoke data access and display the status of your request in Atlan."}
{"id":8629,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You will need to: Integrate ServiceNow and Atlan. Link your individual ServiceNow account to Atlan."}
{"id":8630,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Create a data access approval workflow to enable or revoke access everywhere using ServiceNow. Specify the request state(s) for approval while creating the data access workflow."}
{"id":8631,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Your users will be granted access or their access will be revoked once the request is approved in ServiceNow. Integrate ServiceNow and Atlan."}
{"id":8632,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Link your individual ServiceNow account to Atlan. Create a data access approval workflow to enable or revoke access everywhere using ServiceNow."}
{"id":8633,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Specify the request state(s) for approval while creating the data access workflow. Your users will be granted access or their access will be revoked once the request is approved in ServiceNow."}
{"id":8634,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Specify the request state(s) for approval while creating the data access workflow. Your users will be granted access or their access will be revoked once the request is approved in ServiceNow."}
{"id":8635,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Trigger a webhook - allow requesters to request or revoke data access for any tool. Atlan will trigger a webhook to a URL of your choice for your team to grant or revoke data access."}
{"id":8636,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"For URL , enter the URL for where you want to receive events, including details on requester, approver, and asset, and then validate the URL. danger Atlan will send a sample payload to test if the webhook URL is correct."}
{"id":8637,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"You must respond with a 2xx status for the validation to succeed. Atlan will also run this validation before you save your webhook as a precautionary measure."}
{"id":8638,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Copy the Secret Key and store it in a secure location to verify data access approval or revocation requests from Atlan. For URL , enter the URL for where you want to receive events, including details on requester, approver, and asset, and then validate the URL."}
{"id":8639,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"danger Atlan will send a sample payload to test if the webhook URL is correct. You must respond with a 2xx status for the validation to succeed."}
{"id":8640,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Atlan will also run this validation before you save your webhook as a precautionary measure. For URL , enter the URL for where you want to receive events, including details on requester, approver, and asset, and then validate the URL."}
{"id":8641,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Atlan will send a sample payload to test if the webhook URL is correct. You must respond with a 2xx status for the validation to succeed."}
{"id":8642,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Atlan will also run this validation before you save your webhook as a precautionary measure. Copy the Secret Key and store it in a secure location to verify data access approval or revocation requests from Atlan."}
{"id":8643,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Copy the Secret Key and store it in a secure location to verify data access approval or revocation requests from Atlan. Policy approval You must enable the policy center module to use the policy approval workflow template."}
{"id":8644,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"This template allows you to automate approvals for your data governance policies in Atlan. Automated policy approval workflows can help you streamline the approval process, facilitate compliance with regulatory standards, and simplify data governance for your organization."}
{"id":8645,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Use cases include requests to: Create new policies Revise existing policies Enable governance workflows and inbox You must be an admin user in Atlan to enable the governance workflows and inbox module for your organization. To enable governance workflows and inbox for your Atlan users: From the left menu of any screen in Atlan, click Admin."}
{"id":8646,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Under the Workspace heading, click Labs. On the Labs page, under Governance center , turn on Governance Workflows and Inbox to govern your assets and manage alerts, approvals, and tasks in Atlan more effectively."}
{"id":8647,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"If you'd like to disable the Governance Workflows and Inbox module from your organization's Atlan workspace, follow the steps above to turn it off. Once enabled, you can also temporarily disable the module and turn it on again as needed."}
{"id":8648,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"For any governance workflows you may have created or existing requests , this will not result in any data loss. Interactions with existing access control mechanisms Once you have turned on governance workflows and inbox, the module will interact with existing access control mechanisms in Atlan as follows: Requests : Atlan will channel requests and approvals through governance workflows and land them in the inbox."}
{"id":8649,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"New requests - once you have enabled governance workflows and inbox, the requests widget will be replaced by an inbox and your member and guest users will not be able to raise any new requests until an admin user has created at least one governance workflow. To enable your member and guest users to raise new requests in Atlan: Create a change management governance workflow."}
{"id":8650,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Select all connections present in your Atlan workspace. Skip auto-approval."}
{"id":8651,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Select Anyone approves and list the users or groups designated as your Atlan admins. Publish your first governance workflow!"}
{"id":8652,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Once published, this comprehensive workflow will allow your member and guest users to raise requests. Now you can focus on creating more use-case-driven workflows and consequently removing governed assets from the first workflow until you no longer need it."}
{"id":8653,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Existing requests - only admin users can take action on existing requests from the requests center. Your member and guest users will only be able to raise new requests on governed assets."}
{"id":8654,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"New requests - once you have enabled governance workflows and inbox, the requests widget will be replaced by an inbox and your member and guest users will not be able to raise any new requests until an admin user has created at least one governance workflow. To enable your member and guest users to raise new requests in Atlan: Create a change management governance workflow."}
{"id":8655,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Select all connections present in your Atlan workspace. Skip auto-approval."}
{"id":8656,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Select Anyone approves and list the users or groups designated as your Atlan admins. Publish your first governance workflow!"}
{"id":8657,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Once published, this comprehensive workflow will allow your member and guest users to raise requests. Now you can focus on creating more use-case-driven workflows and consequently removing governed assets from the first workflow until you no longer need it."}
{"id":8658,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Create a change management governance workflow. Select all connections present in your Atlan workspace."}
{"id":8659,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Skip auto-approval. Select Anyone approves and list the users or groups designated as your Atlan admins."}
{"id":8660,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Publish your first governance workflow! Once published, this comprehensive workflow will allow your member and guest users to raise requests."}
{"id":8661,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Now you can focus on creating more use-case-driven workflows and consequently removing governed assets from the first workflow until you no longer need it. Existing requests - only admin users can take action on existing requests from the requests center."}
{"id":8662,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Your member and guest users will only be able to raise new requests on governed assets. Metadata policies - your users must have read access to an asset for triggering governance workflows."}
{"id":8663,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"If an asset is governed by a governance workflow, your users will be able to raise a request on that asset regardless of all allow\/deny permissions in metadata policies. Data policies : No data policy exists - if the workflow connection allows querying and previewing sample data but a data policy has not been configured, your users will be able to raise a data access request on governed assets in the connection."}
{"id":8664,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Data policy with explicit restrictions - if an existing data policy denies querying and previewing sample data and assets are governed by a governance workflow, your users will not be able to raise a data access request on governed assets in the connection. Data policy with explicit grants - if an existing data policy allows querying and previewing sample data and assets are governed by a governance workflow, your users will be able to raise a data access request on governed assets in the connection."}
{"id":8665,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"No data policy exists - if the workflow connection allows querying and previewing sample data but a data policy has not been configured, your users will be able to raise a data access request on governed assets in the connection. Data policy with explicit restrictions - if an existing data policy denies querying and previewing sample data and assets are governed by a governance workflow, your users will not be able to raise a data access request on governed assets in the connection."}
{"id":8666,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Data policy with explicit grants - if an existing data policy allows querying and previewing sample data and assets are governed by a governance workflow, your users will be able to raise a data access request on governed assets in the connection. Domain policies - governance workflows are currently not applicable to domain policies."}
{"id":8667,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"User roles - if an asset is governed by a governance workflow, your users will be able to raise a request on that asset regardless of their role or permissions. For any asset not governed by a governance workflow, default role permissions will apply."}
{"id":8668,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/stewardship\/how-tos\/automate-data-governance","title":"Automate data governance | Atlan Documentation","text":"Connection admins - if an asset is governed by a governance workflow, connection admins will have to go through the approval process for governed assets in the connection. Add, update, and remove resources Add a README to a term using Atlan AI Bulk updates through spreadsheet tools Bulk updates using playbooks Bulk updates using Atlan AI Bulk updates through API, SDK, and CLI operations Metadata updates in supported tools using Atlan browser extension Enable governance workflows and inbox"}
{"id":8669,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8670,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8671,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8672,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8673,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8674,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8675,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8676,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8677,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8678,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#asset-profile","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8679,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Every asset in Atlan has its own asset profile, which consists of all the information available for that particular asset. After you've discovered an asset, click to open the asset profile."}
{"id":8680,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"This view gives you all the context you need about the asset. Components of an asset profile This section displays important details about the asset: Technical name and alias , if added Number of rows and columns Description of the asset Certification status (verified, draft, or deprecated) Owner of the asset For table profiles, you can also view the Columns tab."}
{"id":8681,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"This tab allows you to update the metadata for your columns directly from the asset profile. Lineage offers a visual representation of the sources and transformations of your asset."}
{"id":8682,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"You can also download lineage in a CSV file or as an image from here. Related assets Related assets displays all the assets that have associations with each other beyond a parent-child relationship."}
{"id":8683,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"For example, a Microsoft Power BI workspace will generally have reports, datasets, dashboards, and dataflows, all of which are related to each other. Related assets are additionally listed in the Relations tab of the asset sidebar."}
{"id":8684,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Column preview Column preview offers a snapshot of all the columns in a data table, including the column name, data type, and description. This will change to Field preview in the asset profile and Fields tab in the asset sidebar for certain asset types - including but not limited to data source fields and calculated fields for Tableau data sources, columns for Microsoft Power BI tables, fields for Looker explores and views, and more."}
{"id":8685,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"You can also export child assets such as columns or fields from the parent asset profile. Sample data Sample data shows the sample data for an asset."}
{"id":8686,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"This helps users understand what kind of data is included in the asset and allows them to copy or export this data. Connection admins can also enforce users to validate their credentials before viewing sample data, helping you enforce better governance across your organization."}
{"id":8687,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Linked queries Linked queries displays any saved queries auto-linked to the asset when queried or referenced in the SQL query. This helps users quickly find the saved queries for additional context or launch the query in Insights directly from the asset profile."}
{"id":8688,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"From the Linked queries tab: View all saved queries linked to the asset, along with a total count of such queries. Hover over a linked query to: View the total number of query runs in the last 30 days in a popover."}
{"id":8689,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Click the play icon to run the query in Insights. Click the open asset sidebar icon to view details about the query in the sidebar."}
{"id":8690,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"View the total number of query runs in the last 30 days in a popover. Click the play icon to run the query in Insights."}
{"id":8691,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Click the open asset sidebar icon to view details about the query in the sidebar. Click the 3-dot icon to view any additional linked queries."}
{"id":8692,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"If you do not have access to query data in Atlan, the linked queries will be displayed with a lock icon. Readme provides contextual information about the asset."}
{"id":8693,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"It's a great place to crowdsource all the tribal knowledge and context that different users might have about the data asset. Asset profile header This section helps you perform quick actions."}
{"id":8694,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"From the top right of the asset profile: Click the user avatars to view a list of recently visited users, total views on your asset, total number of unique visitors, and total views by user. Use the days filter to filter asset views and user activity in the last 7, 30, and 90 days."}
{"id":8695,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"This feature is turned on by default - admins can turn off user activity. Use the days filter to filter asset views and user activity in the last 7, 30, and 90 days."}
{"id":8696,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"This feature is turned on by default - admins can turn off user activity. Click the star button to star your asset and bookmark it for easy access."}
{"id":8697,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Expand the Query dropdown to view sample data or query the asset in Insights. Click the clipboard icon to copy the link for your asset."}
{"id":8698,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Click the Slack or Teams icon to post on a Slack or Microsoft Teams channel. Click the 3-dot icon to add an announcement or a resource to your asset."}
{"id":8699,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Asset sidebar The sidebar to the right of the asset profile provides high-level information about the asset. Here's what you can view: Relations shows a list of all the related assets."}
{"id":8700,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Usage displays usage metadata for your Snowflake and Google BigQuery assets. Lineage shows the upstream sources and downstream transformations for the asset."}
{"id":8701,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Fact-Dim Relations displays foreign-key relationships between fact and dimension tables. Fact-dimension relationships between assets can currently only be defined and published via API."}
{"id":8702,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Activity serves as a changelog for the asset. Resources are links to internal or external URLs that help your team understand the asset."}
{"id":8703,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Queries shows all the saved queries for the asset. Requests for an asset can be filtered by their status, such as Pending , Approved , and Rejected."}
{"id":8704,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Properties shows the unique identification number of the asset and other essential properties. Integrations show Slack messages and Jira tickets pertaining to the asset."}
{"id":8705,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/concepts\/what-are-asset-profiles","title":"What are asset profiles? | Atlan Documentation","text":"Custom metadata tabs display custom metadata properties of the asset, if enabled. Components of an asset profile"}
{"id":8706,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8707,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8708,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8709,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8710,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8711,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8712,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8713,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8714,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8715,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#glossary","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8716,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Using familiar terminology helps people quickly understand the data and its context. This is a crucial element of data governance since it adds business context to the data initiatives of an organization."}
{"id":8717,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Why do I need a glossary? In today's diverse data teams, which include people from different backgrounds and use cases, not all of them think about their data in the same way."}
{"id":8718,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"For teams made up of data analysts, data engineers, data scientists, and decision makers, having a shared language is an important step towards ensuring better collaboration. Building a glossary allows your team to define the metrics, columns, and assets with the same meaning for everyone."}
{"id":8719,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Highlights of the Atlan glossary Here's how the Atlan glossary can help your organization: Supports automated metadata management through auto-glossary suggestions from the Atlan bot Anatomy of the Atlan glossary This structure allows for glossaries from multiple domains. A term is the lowest unit that is unique to each glossary."}
{"id":8720,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"It describes the content of the data assets in a useful and precise way. It can exist independently, without belonging to any particular category or subcategory."}
{"id":8721,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Subcategories can be added within categories to provide more context in a glossary. Related term Similar in definition - serves the purpose of a \"see also\" section in a dictionary."}
{"id":8722,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Client is a related term for Customer. Recommended term Preferred form of usage for the current term applied."}
{"id":8723,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"User may be preferred over Customer in the context of your organization. Interchangeable in meaning as another term."}
{"id":8724,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Glossary and Dictionary , or Client and Customer. Opposite in meaning to a particular term."}
{"id":8725,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Minimum is an antonym for the term Maximum , or Loss and Profit are antonyms. Translated term Translated version of the same term in additional languages."}
{"id":8726,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Cliente is the Spanish term for Customer. Valid values for Defines values that are considered appropriate for a related term."}
{"id":8727,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Red , Green , Blue , and Yellow are valid values for the term Color. Classifies and Classified by Country classifies United States , while United States is classified by Country."}
{"id":8728,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/concepts\/what-is-a-glossary","title":"A Glossary | Atlan Documentation","text":"Why do I need a glossary? Highlights of the Atlan glossary Anatomy of the Atlan glossary"}
{"id":8729,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Atlan allows you to add owners for each data asset. This enriches the asset profile and helps build trust among users."}
{"id":8730,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Users can then quickly reach out to the owner of the asset for any questions about the data. The owner is responsible for maintaining the data asset."}
{"id":8731,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"They are the right person to contact for questions about the data's frequency of update, progress, status, and more. Add owners to your assets To add or update owners for a data asset, follow these steps: On the Atlan homepage, click Assets in the left menu."}
{"id":8732,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Click on an asset to view its asset profile. To assign owners, you can either: Click the checkbox next to an individual user's name."}
{"id":8733,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Toggle to groups in the top right and select an entire group of users. Click the checkbox next to an individual user's name."}
{"id":8734,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Toggle to groups in the top right and select an entire group of users. Click Save."}
{"id":8735,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Your asset profile will now display the asset owners! ð You can also filter your assets by asset owners."}
{"id":8736,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"Here are the steps: In the Filters menu on the Assets page, click Owners. Click the checkbox next to an owner name to view their owned assets."}
{"id":8737,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-owners","title":"Add owners | Atlan Documentation","text":"You can select No Owners in the Owners filter to view assets that currently do not have assigned owners and assign accordingly if needed. Add owners to your assets"}
{"id":8738,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"How many times has someone complained to you that the data is incomplete or has issues? And how many times have you responded that it's still a work in progress or they're using the wrong data!"}
{"id":8739,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"Wouldn't it be really convenient if the data could answer these questions? Certificates in Atlan can help!"}
{"id":8740,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"The certification tags help users quickly identify whether a data asset is ready to use, a work in progress, or has some issues. You can add the following four certification tags to any data asset: Verified for ready to be used."}
{"id":8741,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"Draft for work in progress. Deprecated if the asset no longer exists."}
{"id":8742,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"No certificate if not required or needs documentation down the line. Certificates can be used to quickly filter data assets on the Assets page."}
{"id":8743,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"This helps build trust in your data assets among users. Add certificates to your assets To add or update the certificate for your data assets, follow these steps: On the Atlan homepage, click Assets in the left menu."}
{"id":8744,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"Click on the asset to open its asset profile. In the right menu, click + under Certificate and choose the relevant certification option."}
{"id":8745,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"Write a message to add more context. Now your data can proudly display its status for all to see!"}
{"id":8746,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/add-certificates","title":"Add certificates | Atlan Documentation","text":"ð Once you have selected a certification tag for your data asset, you will get a popup that your certificate has been saved. Add certificates to your assets"}
{"id":8747,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Provide additional context for your assets to other users in your organization. Create common definitions once and apply them many times to multiple assets."}
{"id":8748,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Offer an abstract point for applying tags to be propagated to all linked assets - including their downstream and child assets - if propagation is enabled. If your data assets include personal information - for example, email addresses - you can link your assets to an Email Address term to provide context to your users."}
{"id":8749,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"You can define the term Email Address once in the glossary. You can link the term to all the columns where an individual's email address appears."}
{"id":8750,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"You can also tag the term as PII - and all of the linked assets will be tagged as PII. To link a term to an asset: From a term From the left menu on any screen, click Glossary."}
{"id":8751,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Under Glossary in the left menu, click the name of your glossary. Under your glossary name, click the category in which your term is nested and then click the term you would like to link to your assets."}
{"id":8752,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Click + Link Assets to get started. In the sidebar on the right, select the asset(s) to which you would like to link the term."}
{"id":8753,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"At the bottom of the sidebar, click Link asset(s) to confirm your selections. From an asset From the left menu on any screen, click Assets."}
{"id":8754,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"On the Assets page, select the asset to which you would like to link a term. In the dialog, expand the glossary menu and then click the term you would like to link to your assets."}
{"id":8755,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Click Save to confirm your selections. You can now view linked assets for your glossary term!"}
{"id":8756,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"ð To unlink a term from an asset: From a term From the left menu on any screen, click Glossary. Under Glossary in the left menu, click the name of your glossary."}
{"id":8757,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"Under your glossary name, click the category in which your term is nested and then click the term you would like to unlink from your assets. Under Linked assets , navigate to the asset(s) from which you would like to unlink the term."}
{"id":8758,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"To the right of the asset name, click the three dots and then click Unlink asset. From an asset From the left menu on any screen, click Assets."}
{"id":8759,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/glossary\/how-tos\/link-terms-to-assets","title":"Link terms to assets | Atlan Documentation","text":"On the Assets page, select the asset from which you would like to unlink a term. Your assets will now be unlinked from the glossary term."}
{"id":8760,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Atlan is a living catalog of all your data assets and knowledge. It lets you quickly discover and access your data, along with the tribal knowledge and business context."}
{"id":8761,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Certify your assets Enrich your assets with descriptions Star your assets for easy access Intelligent keyword recognition Click Assets in the left-side panel. Domains : Filter assets by domains , such as a single domain, multiple domains, or no domain."}
{"id":8762,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Owners : Filter by selecting one or more users. You can also toggle between users and groups to filter based on a group of users."}
{"id":8763,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Tags : Filter by user-generated tags, such as public , PII , and more. Name : Sort by the asset name in an alphabetical or a reverse alphabetical order."}
{"id":8764,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Updated on Atlan : Sort by the newest or oldest updated assets. Star count : Sort assets by most or fewest stars."}
{"id":8765,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"Popularity : Sort Snowflake and Google BigQuery assets by the most or least popular assets. The sorting options may vary depending on the asset type selected."}
{"id":8766,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/discovery\/how-tos\/search-and-discover-assets","title":"Search and discover assets | Atlan Documentation","text":"For example, if you are viewing the results while filtering by the Table tab, you'll also have the option of sorting by the most or fewest number of rows and columns. See only what you want to see"}
{"id":8767,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization."}
{"id":8768,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center. You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations."}
{"id":8769,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access. Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have."}
{"id":8770,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data."}
{"id":8771,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience."}
{"id":8772,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution."}
{"id":8773,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data."}
{"id":8774,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used."}
{"id":8775,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed."}
{"id":8776,"url":"https:\/\/docs.atlan.com\/get-started\/how-tos\/quick-start-for-admins#discovery","title":"Administrators | Atlan Documentation","text":"Sort by popularity to quickly discover what assets your teammates are using every day."}
{"id":8777,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8778,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8779,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8780,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8781,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8782,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8783,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8784,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8785,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8786,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8787,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8788,"url":"https:\/\/docs.atlan.com\/secure-agent#__docusaurus_skipToContent_fallback","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8789,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8790,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8791,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8792,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8793,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8794,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8795,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8796,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8797,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8798,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8799,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8800,"url":"https:\/\/docs.atlan.com\/secure-agent#key-capabilities","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8801,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8802,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8803,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8804,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8805,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8806,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8807,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8808,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8809,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8810,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8811,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8812,"url":"https:\/\/docs.atlan.com\/secure-agent#security-first-architecture","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8813,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8814,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8815,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8816,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8817,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8818,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8819,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8820,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8821,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8822,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8823,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8824,"url":"https:\/\/docs.atlan.com\/secure-agent#scalable-metadata-extraction","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8825,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8826,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8827,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8828,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8829,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8830,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8831,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8832,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8833,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8834,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8835,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8836,"url":"https:\/\/docs.atlan.com\/secure-agent#flexible-deployment","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8837,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8838,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8839,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8840,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8841,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8842,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8843,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8844,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8845,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8846,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8847,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8848,"url":"https:\/\/docs.atlan.com\/secure-agent#automated-operations","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8849,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8850,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8851,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8852,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8853,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8854,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8855,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8856,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8857,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8858,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8859,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8860,"url":"https:\/\/docs.atlan.com\/secure-agent#how-it-works","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8861,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity."}
{"id":8862,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway."}
{"id":8863,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary."}
{"id":8864,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance."}
{"id":8865,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs."}
{"id":8866,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters."}
{"id":8867,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan."}
{"id":8868,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts."}
{"id":8869,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job."}
{"id":8870,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion."}
{"id":8871,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing."}
{"id":8872,"url":"https:\/\/docs.atlan.com\/secure-agent#see-also","title":"Secure Agent | Atlan Documentation","text":"See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works"}
{"id":8873,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"The Atlan Secure Agent is a Kubernetes-based application that runs within a customer's environment. It acts as a gateway between the single-tenant Atlan SaaS and external systems like Snowflake, Tableau, and other data sources."}
{"id":8874,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"This document explains the Secure Agent's deployment architecture, key components, communication flows, and security considerations. High-level architecture This section describes how the Secure Agent is structured and deployed."}
{"id":8875,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"It explains the core components that enable metadata extraction, job execution, and communication with Atlan. Figure 1: Atlan Secure Agent deployment architecture."}
{"id":8876,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Core components The Secure Agent runs as a Kubernetes-based application within a customer's private cloud or on-premises environment. It consists of several key components that work together to execute metadata extraction tasks."}
{"id":8877,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Argo Workflows An Argo Workflow server is deployed to coordinate all activities and launch Kubernetes workloads. The Secure Agent uses Argo Workflows to orchestrate and manage metadata extraction jobs."}
{"id":8878,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Each workflow represents a unit of work, such as extracting metadata from a source system. Agent orchestrator A scheduled job that runs every five minutes to check for jobs that need to be executed."}
{"id":8879,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"It connects to the Atlan tenant, retrieves job details, and initiates workflows accordingly. Auxiliary services Additional services that support agent operations: Health monitoring service sends periodic heartbeats to Atlan to confirm the agent is active."}
{"id":8880,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Logging service uploads execution logs to Atlan for monitoring and debugging. Metadata extraction workflows Connector-specific jobs that extract metadata from source systems."}
{"id":8881,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Workflows run in isolated containers, ensuring security and reliability. Data flow The Secure Agent supports two modes of metadata transfer."}
{"id":8882,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Each mode determines how extracted metadata is delivered to Atlan. Bucket relay Metadata extraction in bucket relay mode stores metadata in enterprise-managed cloud storage before Atlan retrieves it."}
{"id":8883,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Figure 2: Data flow in bucket relay mode. The Secure Agent extracts metadata and writes it to a storage bucket in the customer s cloud environment (such as AWS S3, Azure Blob Storage, or Google Cloud Storage)."}
{"id":8884,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"This is managed by providing the agent write access to cloud storage. Atlan retrieves metadata from the storage bucket and processes it further."}
{"id":8885,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"This is managed by providing Atlan read access to list and read files in cloud storage. This mode ensures the extracted data remains within the customer s infrastructure until Atlan explicitly fetches it."}
{"id":8886,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Customers can also use this data for auditing. Direct ingestion In direct ingestion mode, metadata is transferred directly from the Secure Agent to Atlan."}
{"id":8887,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Figure 3: Data flow in direct ingestion mode. The Secure Agent uses pre-signed URLs to upload metadata directly to Atlan."}
{"id":8888,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Some cloud storage providers that use pre-signed URLs include: The Secure Agent uses pre-signed URLs to upload metadata directly to Atlan. Some cloud storage providers that use pre-signed URLs include: A pre-signed URL grants temporary access to upload files without exposing long-term credentials."}
{"id":8889,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"A pre-signed URL grants temporary access to upload files without exposing long-term credentials. Each URL has an expiration time, ensuring access is only available for a limited duration."}
{"id":8890,"url":"https:\/\/docs.atlan.com\/secure-agent\/references\/deployment-architecture","title":"Deployment architecture | Atlan Documentation","text":"Each URL has an expiration time, ensuring access is only available for a limited duration. See also Secure Agent - Security : Details on security mechanisms."}
{"id":8891,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#__docusaurus_skipToContent_fallback","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":8892,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#__docusaurus_skipToContent_fallback","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":8893,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#get-started","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":8894,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#get-started","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":8895,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"ð ¤ Who can do this? You will need to be an admin user in Atlan to create playbooks."}
{"id":8896,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"A common question that data teams often face is how to automate metadata at scale. Having started out as a data team ourselves, we know that automating repetitive tasks can help data teams maximize the value they provide to their organization."}
{"id":8897,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"One way of doing so is through Atlan's playbooks! Playbooks help power metadata automation for your data assets in Atlan."}
{"id":8898,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"You can create rule-based automations at scale and update metadata in bulk, helping you streamline your workflows. You can update the following asset metadata using playbooks: For example, imagine your organization needs to transfer ownership of several data assets."}
{"id":8899,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Instead of your data team manually updating the ownership of each and every asset, you can create a playbook to automate this process and update the metadata of your assets at scale. Playbook recommendations Before you begin, review some general guidelines on running playbooks in Atlan: Avoid running multiple playbooks simultaneously on the same set of assets."}
{"id":8900,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Allow one playbook run to be completed before proceeding with another operation on the same set of assets. Otherwise, you may experience performance issues and inconsistencies."}
{"id":8901,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Review and understand the depth of your asset lineage or hierarchy prior to enabling a tag propagation playbook. For assets with complex lineage, tag propagation may take longer to complete than the playbook runtime."}
{"id":8902,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"You may want to review and judiciously select a list of assets that need to be tagged directly. For their child and\/or downstream assets, Atlan recommends that you enable tag propagation."}
{"id":8903,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Create a playbook To create a playbook in Atlan: From the Filters menu on the left or the tabs along the top, apply any asset filters. Click Governance to navigate to the governance center."}
{"id":8904,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Under the Governance heading of the Governance center , click Playbooks. Click Create New to get started."}
{"id":8905,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Under the Governance heading of the Governance center , click Playbooks. Click Create New to get started."}
{"id":8906,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"In the Create new playbook dialog box, enter the following details: For Name , enter a name for the task to be accomplished - for example, Update ownership. (Atlan recommends that the length of a playbook name must be no longer than 46 characters.) (Optional) For Description , enter a description."}
{"id":8907,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) Select an icon for your playbook. For Name , enter a name for the task to be accomplished - for example, Update ownership."}
{"id":8908,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Atlan recommends that the length of a playbook name must be no longer than 46 characters.) (Optional) For Description , enter a description. (Optional) Select an icon for your playbook."}
{"id":8909,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Create to save your playbook. Set up rules as filters To set up rules as filters for your playbook: In the Build Rules page of your playbook, click Filters."}
{"id":8910,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"For name, add a name to your filter. To set a matching condition for the filters, select Match all or Match any."}
{"id":8911,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Match all will logically AND the criteria, while Match any will logically OR the criteria. For this example, we'll click Connection and then select a Snowflake connection."}
{"id":8912,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) To further refine your asset selection: Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection."}
{"id":8913,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection."}
{"id":8914,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Connector to filter assets by supported connectors. Click Asset type to filter by specific asset types - for example, tables, columns, queries, glossaries, and more."}
{"id":8915,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Certificate to filter assets by certification status. Click Owners to filter assets by asset owners."}
{"id":8916,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Tags to filter assets by your tags in Atlan, including imported Snowflake and dbt tags. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value."}
{"id":8917,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. Click Domains to filter by specific domains or subdomains to bulk update all the assets included in those data domains or subdomains."}
{"id":8918,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Products to filter for data products by specific data domains or subdomains. Click Schema qualified Name to filter assets by the qualified name of a given schema."}
{"id":8919,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Database qualified Name to filter assets by the qualified name of a given database. Click dbt to filter assets by dbt-specific filters and then select a dbt Cloud or dbt Core filter."}
{"id":8920,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Properties to filter assets by common asset properties. Click Usage to filter assets by usage metrics."}
{"id":8921,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Monte Carlo to filter assets by Monte Carlo-specific filters. Click Soda to filter assets by Soda-specific filters."}
{"id":8922,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Table\/View to filter tables or views by row count, column count, or size. Click Column to filter columns by column-specific filters , including parent asset type or name, data type, or column keys."}
{"id":8923,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Process to filter lineage processes by the SQL query. Click Query to filter assets by associated visual queries."}
{"id":8924,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Measure to filter Microsoft Power BI measures using the external measures filter. Select Starts With or Ends With to filter assets using the starting or ending sequence of values."}
{"id":8925,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Select Contains or Does not contain to find assets with or without specified values contained within the attribute. Select Pattern to filter assets using supported Elastic DSL regular expressions."}
{"id":8926,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Select Is empty to filter assets with null values. Select Belongs to or Doesn't belong to to filter data products by specific data domains or subdomains."}
{"id":8927,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"For Values , select the relevant values. The values will vary depending on the selected attributes."}
{"id":8928,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) To add more filters, click Add filter and select Filter to add individual filters or Filter Group to nest more filters in a group. (Optional) To view all the assets that match your rules, in the Filters card, click View all for a preview."}
{"id":8929,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) To remove a playbook filter, to the right of any filter, click the three horizontal dots and then click Delete. (Optional) To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable."}
{"id":8930,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Enable to turn on any disabled filters. Select the actions To select the actions to be performed based on your rules: In the Build Rules page of your playbook, click Actions."}
{"id":8931,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Certificate to update the certification status of assets to Verified , Draft , Deprecated , or No certificate. Click Description to update the description of your assets."}
{"id":8932,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Owners to add, remove, or replace asset owners. In this example, we'll update the ownership of the assets."}
{"id":8933,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Tags to add tags to your assets or remove or replace them from tagged or propagated assets. Note that if there are multiple tag actions to be performed, Atlan will execute them in the following order: ADD , REMOVE , and then REPLACE."}
{"id":8934,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Domain to add your assets to a specific domain or subdomain or remove them from an existing linked domain or subdomain. Click any custom metadata structure and then select a custom metadata property to update or unlink it from your assets."}
{"id":8935,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"For Select operator , select the relevant option. The operators will vary depending on the selected action."}
{"id":8936,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"For Values , select the relevant option(s). The values will vary depending on the selected actions."}
{"id":8937,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"(Optional) To add more actions, click Add Action. You can control tag propagation when adding tags as an action in playbooks."}
{"id":8938,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Tag propagation is disabled by default. If you enable tag propagation, you will also be able to configure how tags are propagated."}
{"id":8939,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Run the playbook If you'd like to continue working on your playbook, you can save it as a draft. If your playbook is ready, you can proceed to running it."}
{"id":8940,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"To run the playbook: You can either: To run the playbook once immediately, click Run once. To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time."}
{"id":8941,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"danger If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs. For more about workflows in general, see workflow recommendations."}
{"id":8942,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"You can either: To run the playbook once immediately, click Run once. To run the playbook once immediately, click Run once."}
{"id":8943,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. danger If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs."}
{"id":8944,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"For more about workflows in general, see workflow recommendations. To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time."}
{"id":8945,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs. For more about workflows in general, see workflow recommendations."}
{"id":8946,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Click Complete to run the playbook. Click Complete to run the playbook."}
{"id":8947,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"In the resulting screen, click Go to profile to view your playbook profile. In the resulting screen, click Go to profile to view your playbook profile."}
{"id":8948,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Once your playbook has completed its run, you will see the metadata updated for your assets! ð If you have any questions about setting up playbooks, head over here."}
{"id":8949,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/set-up-playbooks","title":"Set up playbooks | Atlan Documentation","text":"Create a playbook Set up rules as filters Select the actions Run the playbook"}
{"id":8950,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#guides","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":8951,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#guides","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":8952,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#playbook-management","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":8953,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#playbook-management","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":8954,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Once you've created a playbook , you can monitor, modify, or delete it at any time. You can also enable notifications to monitor your playbook runs directly in Slack or Microsoft Teams."}
{"id":8955,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Monitor a playbook To monitor your playbooks runs: When running a playbook immediately, you will be redirected to the monitoring page within 5 seconds. From the left menu of any screen in Atlan, click Governance."}
{"id":8956,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Under the Governance heading of the Governance center , click Playbooks. In the playbooks manager, select the playbook you'd like to view."}
{"id":8957,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"A summary of the rules, actions, and updated assets. An activity log for recent runs and updates over time."}
{"id":8958,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"The activity log in the playbooks manager can help you keep track of playbook runs, ranging from 24 hours to 30 days. Select any of the entries to navigate to the corresponding playbook."}
{"id":8959,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Modify a playbook To modify an existing playbook: From the left menu of any screen in Atlan, click Governance. Under the Governance heading of the Governance center , click Playbooks."}
{"id":8960,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"In the playbooks manager, click the playbook you'd like to modify. On your playbook page: To edit the name and description of your playbook, hover over your playbook and click Edit."}
{"id":8961,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To modify the rules of your playbook, click Rules to make your changes and then click Update to save them. (Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule."}
{"id":8962,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable. Click Enable to turn on any disabled filters."}
{"id":8963,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To modify the schedule for your playbook, in the upper right of your screen: Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule."}
{"id":8964,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To edit the name and description of your playbook, hover over your playbook and click Edit. To modify the rules of your playbook, click Rules to make your changes and then click Update to save them."}
{"id":8965,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"(Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule. (Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule."}
{"id":8966,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable. Click Enable to turn on any disabled filters."}
{"id":8967,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To modify the schedule for your playbook, in the upper right of your screen: Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule."}
{"id":8968,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule."}
{"id":8969,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Delete a playbook To delete an existing playbook: From the left menu of any screen in Atlan, click Governance. Under the Governance heading of the Governance center , click Playbooks."}
{"id":8970,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"In the playbooks manager, hover over the playbook you'd like to delete and click Delete Playbook. Click Delete to confirm."}
{"id":8971,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Enable playbook notifications You can set up Slack or Microsoft Teams alerts for your playbook runs in Atlan. This can help you monitor your playbooks directly in Slack or Microsoft Teams."}
{"id":8972,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"You can also choose to receive alerts for failed playbook runs only. Before you can enable notifications for playbooks, you will need to either: Integrate Slack and Atlan Integrate Microsoft Teams and Atlan To enable notifications for playbook runs: From the left menu of any screen in Atlan, click Governance."}
{"id":8973,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Under the Governance heading of the Governance center , click Playbooks. In the upper-right of the playbooks manager, under Activity , click the bell icon."}
{"id":8974,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"In the Enable notifications popup: Click Setup now to integrate Slack or Microsoft Teams. If you have already integrated Slack or Microsoft Teams, click Enable."}
{"id":8975,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Click Setup now to integrate Slack or Microsoft Teams. If you have already integrated Slack or Microsoft Teams, click Enable."}
{"id":8976,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"In the notifications setup dialog, configure the following: For Notifications channel , you can either: If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected. You can use the same channel to receive both workflow and playbook run alerts and skip to the next step."}
{"id":8977,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs. This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration."}
{"id":8978,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"To select the type of notifications you want to receive, you can either: Click Both success and failure alerts to receive notifications for both successful and failed playbook runs. Click Failure alerts only to limit notifications to failed playbook runs only."}
{"id":8979,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Click Save to save your notification preferences. For Notifications channel , you can either: If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected."}
{"id":8980,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"You can use the same channel to receive both workflow and playbook run alerts and skip to the next step. If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs."}
{"id":8981,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration. If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected."}
{"id":8982,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"You can use the same channel to receive both workflow and playbook run alerts and skip to the next step. If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs."}
{"id":8983,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration. To select the type of notifications you want to receive, you can either: Click Both success and failure alerts to receive notifications for both successful and failed playbook runs."}
{"id":8984,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Click Failure alerts only to limit notifications to failed playbook runs only. Click Both success and failure alerts to receive notifications for both successful and failed playbook runs."}
{"id":8985,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"Click Failure alerts only to limit notifications to failed playbook runs only. Click Save to save your notification preferences."}
{"id":8986,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"(Optional) To disable notifications, from the notifications setup dialog, remove the playbook alerts channel configured for Slack or Microsoft Teams. You will now receive Slack or Microsoft Teams notifications for all your playbook runs in Atlan!"}
{"id":8987,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/manage-playbooks","title":"Manage playbooks | Atlan Documentation","text":"ð The Atlan bot will share playbook run alerts, including details like run status, start time, run time, trigger type, last three runs, and more. Monitor a playbook Modify a playbook Delete a playbook Enable playbook notifications"}
{"id":8988,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Available via the Data Quality Studio package ð ¤ Who can do this? You need to be an admin user in Atlan to create profiling playbooks."}
{"id":8989,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Monitoring and improving data quality is critical to building trust in your data assets. Atlan solves for this with profiling playbooks!"}
{"id":8990,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Profiling playbooks help power data observability for your assets in Atlan. You can create profiling playbooks to scan your assets at scale, identify any issues or inconsistencies, and improve the data quality of your assets."}
{"id":8991,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Supported sources Atlan currently supports column profiling for the following connectors: Microsoft SQL Server Create a profiling playbook To create a profiling playbook: In the left menu in Atlan, click Governance. Under the Governance heading of the Governance center , click Playbooks."}
{"id":8992,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"To the right of the Create New button, click the downward arrow and then select Profiling Playbook. In the Create new profiling playbook dialog, enter the following details: For Name , enter a name for the task to be accomplished - for example, Tables scan."}
{"id":8993,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Atlan recommends that the length of a playbook name must be no longer than 46 characters.) For Connection , select a supported connection from the dropdown menu - in this example, we'll select a Google BigQuery connection development. (Optional) For Description , enter a description for your playbook."}
{"id":8994,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Optional) Select an icon for your playbook. For Name , enter a name for the task to be accomplished - for example, Tables scan."}
{"id":8995,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Atlan recommends that the length of a playbook name must be no longer than 46 characters.) For Connection , select a supported connection from the dropdown menu - in this example, we'll select a Google BigQuery connection development. (Optional) For Description , enter a description for your playbook."}
{"id":8996,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Optional) Select an icon for your playbook. Click Create to save your playbook."}
{"id":8997,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Set up rules as filters The assets to be scanned are pre-filled based on your selected connection. To set up rules as filters for your profiling playbook: In the Build Rules page of your profiling playbook, click Filters."}
{"id":8998,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"For the name field, add a name to your filter - for example, Profiling action. To set a matching condition for the filters, select Match all or Match any."}
{"id":8999,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Match all will logically AND the criteria, while Match any will logically OR the criteria. Click Connection to select a specific connection."}
{"id":9000,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection."}
{"id":9001,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection."}
{"id":9002,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Connector to filter assets by supported connectors. Click Asset type to filter by specific asset types - for example, tables, columns, queries, glossaries, and more."}
{"id":9003,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Certificate to filter assets by certification status. Click Owners to filter assets by asset owners."}
{"id":9004,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Tags to filter assets by your tags in Atlan, including imported Snowflake and dbt tags. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value."}
{"id":9005,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. Click Schema qualified Name to filter assets by the qualified name of a given schema."}
{"id":9006,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Database qualified Name to filter assets by the qualified name of a given database. Click dbt to filter assets by dbt-specific filters and then select a dbt Cloud or dbt Core filter."}
{"id":9007,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Properties to filter assets by common asset properties. Click Usage to filter assets by usage metrics."}
{"id":9008,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Monte Carlo to filter assets by Monte Carlo-specific filters. Click Soda to filter assets by Soda-specific filters."}
{"id":9009,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Table\/View to filter tables or views by row count, column count, or size. Click Column to filter columns by column-specific filters , including parent asset type or name, data type, or column keys."}
{"id":9010,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Process to filter lineage processes by the SQL query. Click Query to filter assets by associated visual queries."}
{"id":9011,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Measure to filter Microsoft Power BI measures using the external measures filter. Select Starts With or Ends With to filter assets using the starting or ending sequence of values."}
{"id":9012,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Select Contains or Does not contain to find assets with or without specified values contained within the attribute. Select Pattern to filter assets using supported Elastic DSL regular expressions."}
{"id":9013,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Select Is empty to filter assets with null values. For Values , select the relevant values."}
{"id":9014,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"The values will vary depending on the selected attributes. (Optional) To add more filters, click Add filter and select Filter to add individual filters or Filter Group to nest more filters in a group."}
{"id":9015,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"(Optional) To view all the assets that match your rules, in the Filters card, click View all for a preview. Confirm profiling actions Column profiling is currently only supported for number and text data types."}
{"id":9016,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"The profiled column assets will be populated with preconfigured metrics. To select the actions to be performed based on your rules: The default profiling actions to be performed include: Base metrics : Distinct count - number of rows that contain distinct values, relative to the column."}
{"id":9017,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Missing count - number of rows that do not contain specific values. Numeric metrics : Minimum and maximum values - smallest and greatest values in a numeric column."}
{"id":9018,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column."}
{"id":9019,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Variance - calculated variance of values in a numeric column. String metrics : Average length - average length of string values in a column."}
{"id":9020,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Minimum and maximum length - minimum and maximum length of string values in a column. Base metrics : Distinct count - number of rows that contain distinct values, relative to the column."}
{"id":9021,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Missing count - number of rows that do not contain specific values. Distinct count - number of rows that contain distinct values, relative to the column."}
{"id":9022,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Missing count - number of rows that do not contain specific values. Numeric metrics : Minimum and maximum values - smallest and greatest values in a numeric column."}
{"id":9023,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column."}
{"id":9024,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Variance - calculated variance of values in a numeric column. Minimum and maximum values - smallest and greatest values in a numeric column."}
{"id":9025,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column."}
{"id":9026,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Variance - calculated variance of values in a numeric column. String metrics : Average length - average length of string values in a column."}
{"id":9027,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Minimum and maximum length - minimum and maximum length of string values in a column. Average length - average length of string values in a column."}
{"id":9028,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Minimum and maximum length - minimum and maximum length of string values in a column. Click Next to proceed to the next step."}
{"id":9029,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"In the Optimize your Profiling query popup, the following message will appear: This Profiling playbook will query x rows across y assets. To avoid significant computing costs, review your applied filters before proceeding."}
{"id":9030,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Review filters to review your existing filters or click Continue anyway to proceed. Note that Atlan is working to support sampling functionality in the future."}
{"id":9031,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Run the playbook If you'd like to continue working on your playbook, you can save it as a draft. If your playbook is ready, you can proceed to running it."}
{"id":9032,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"To run the playbook: You can either: To run the playbook once immediately, click Run once. To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time."}
{"id":9033,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"To run the playbook once immediately, click Run once. To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time."}
{"id":9034,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click Complete to confirm your selections. In the resulting screen, click Go to profile to view your playbook profile."}
{"id":9035,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Once your playbook run is completed, you will see the data profile updated for your assets! ð View profiled assets To view the profiled assets for your playbook: In the sidebar to the right, profiled assets will be indicated with a bar graph icon."}
{"id":9036,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Click any profiled asset to proceed to viewing profiling data. From the table sidebar, click the Column tab to view column assets and then select any of the profiled columns."}
{"id":9037,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"In the column sidebar to the right, click Profile to view profiling data for the selected column asset. Once you've created a profiling playbook, you can monitor, modify, or delete it at any time."}
{"id":9038,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/how-tos\/automate-data-profiling","title":"Automate data profiling | Atlan Documentation","text":"Create a profiling playbook Set up rules as filters Confirm profiling actions Run the playbook View profiled assets"}
{"id":9039,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#troubleshooting","title":"Playbooks | Atlan Documentation","text":"Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks."}
{"id":9040,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks#troubleshooting","title":"Playbooks | Atlan Documentation","text":"Troubleshooting playbooks : Solutions for common playbook issues."}
{"id":9041,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"ð ¤ Who can do this? You will need to be an admin user in Atlan to create playbooks."}
{"id":9042,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Here are a few things to know about setting up playbooks : Following are the known issues or limitations when using the domain action : If you do not have read permission on the assets you want to add to a domain, those assets will be removed from the playbook workflow during processing. If you do not have update permission on the assets you want to add to a domain, the playbook workflow will fail."}
{"id":9043,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"However, some assets may still be linked to the domain before the failure occurs. We recommend building no more than a maximum of 20 rules per playbook."}
{"id":9044,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"However, the total number of playbooks that can be run is still to be determined. From a technical standpoint, playbooks leverage the workflow infrastructure, which means there are no hard limits."}
{"id":9045,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Depending on the number of playbooks that need to be run, the infrastructure will have to be scaled accordingly. Yes."}
{"id":9046,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"You need to have the permission to update assets in Atlan in order to run playbooks for updating them. If you do not have the permission to update an asset, you will be unable to update it using playbooks."}
{"id":9047,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Additionally, Atlan uses the permissions of the playbook creator in determining the assets to be updated and not that of the user who runs the playbook. Your user permissions are used to determine the bulk updates you can make to ensure that there is no adverse impact on assets beyond your scope of access."}
{"id":9048,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"No, Atlan currently does not support automating asset update requests through playbooks. Currently, there is no button to undo asset updates."}
{"id":9049,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"However, you can modify your existing playbooks. You can either turn off the filters or add new rules to reverse the updates."}
{"id":9050,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Currently, no. You can monitor your existing playbooks to view a high-level summary of asset updates from previous playbook runs."}
{"id":9051,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Observability of results is on the roadmap. Currently, no."}
{"id":9052,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"However, you can set up Slack or Microsoft Teams alerts for your playbook runs in Atlan. If you encounter an offload node status is not supported error message, the playbook workflow may have exceeded the EtcD size limit."}
{"id":9053,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"Playbooks use Argo workflow templates, which are stored as Kubernetes resources. This creates a limit to their size."}
{"id":9054,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/playbooks\/troubleshooting\/troubleshooting-playbooks","title":"Troubleshooting playbooks | Atlan Documentation","text":"To handle this error, Atlan recommends the following: Reduce the number of rules in your playbook Optimize filters for asset selection"}
{"id":9055,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts#__docusaurus_skipToContent_fallback","title":"Contracts | Atlan Documentation","text":"Get started Follow these steps to implement contracts in Atlan: Create data contracts Add contract impact analysis in GitHub : Detailed instructions on adding contracts for impact analysis in GitHub."}
{"id":9056,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts#get-started","title":"Contracts | Atlan Documentation","text":"Get started Follow these steps to implement contracts in Atlan: Create data contracts Add contract impact analysis in GitHub : Detailed instructions on adding contracts for impact analysis in GitHub."}
{"id":9057,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"A data contract is an agreement between a data producer and consumer that specifies requirements for generating and using high-quality, reliable data. As a powerful tool for data management, data contracts can help you standardize contractual obligations between data producers and consumers, organize your assets with embeddable contract metadata, and enforce them with data quality rules."}
{"id":9058,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"In Atlan, you can directly add a data contract to supported assets and provide helpful context to your downstream users. For a data contract to help build trust in your assets, it should be: Templatized and easily comprehensible - use Atlan's YAML contract template to create standardized contracts and push to Atlan."}
{"id":9059,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Version-controlled - continuously validate and monitor your data contracts either in runtime or real-time. Embeddable - embed the contract as metadata for a supported asset."}
{"id":9060,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Enforceable - enforce your contracts with data quality rules. Extensible - identify new specifications, generate new versions, and then compare and contrast them."}
{"id":9061,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"You can create webhooks for data contracts and receive notifications for when a contract is added or updated to a URL of your choice. Supported asset types You can create data contracts for the following asset types: Output port assets of data products Supported asset metadata Atlan maps the following asset metadata properties to it contract properties: Add a data contract to an asset Any non-guest user with edit access to an asset's metadata can create, deploy, and manage data contracts."}
{"id":9062,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"This only includes admin and member users in Atlan. To add a data contract to an asset, you can either: Create a contract directly in Atlan from the Contracts tab of the asset profile."}
{"id":9063,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"You can create and maintain data contracts as easily as editing a word document. Use Atlan CLI to import an existing contract from your local machine to Atlan directly or through a CI\/CD pipeline."}
{"id":9064,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Atlan CLI is a command-line tool that you can download directly from Atlan to your local machine to create and push data contracts to Atlan. Once you have published the contract, you can also sync metadata from a contract to the governed asset in Atlan."}
{"id":9065,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Once created, you will be able to monitor and manage your data contracts in Atlan. To add a data contract: From the left menu of any screen in Atlan, click Assets."}
{"id":9066,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"(Optional) From the Filters menu on the left, click Properties and then click Has contract. Click No to filter for assets without a contract."}
{"id":9067,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"From the Assets page, select an asset to open the asset sidebar. In the Contract tab of the asset profile, you can either: Click Create contract to create a draft contract directly in Atlan based on asset metadata."}
{"id":9068,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Click Import contract to use Atlan CLI to import an existing contract from your local environment to Atlan. You will first need to install and connect Atlan CLI and then push the contract to Atlan."}
{"id":9069,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Refer to our developer documentation to complete the steps. Click Create contract to create a draft contract directly in Atlan based on asset metadata."}
{"id":9070,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Click Import contract to use Atlan CLI to import an existing contract from your local environment to Atlan. You will first need to install and connect Atlan CLI and then push the contract to Atlan."}
{"id":9071,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Refer to our developer documentation to complete the steps. (Optional) Click the Edit button to edit the contract."}
{"id":9072,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Congrats on adding a data contract in Atlan! ð View a data contract To view a data contract: From the left menu of any screen in Atlan, click Assets."}
{"id":9073,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"From the Assets page, select an asset to open the asset sidebar. Click the Document icon to open a read-only, simplified view of your contract."}
{"id":9074,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Next to Published version , click the version dropdown to view the latest version of the contract. Select an older version and then click Compare with published version to compare them side by side."}
{"id":9075,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Click the Edit button to edit the contract. Click the clipboard icon to copy the YAML code."}
{"id":9076,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/create-data-contracts","title":"Create data contracts | Atlan Documentation","text":"Under Timeline , view a timeline for the evolution of your contract. Supported asset types Supported asset metadata Add a data contract to an asset View a data contract"}
{"id":9077,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts#guides","title":"Contracts | Atlan Documentation","text":"Get started Follow these steps to implement contracts in Atlan: Create data contracts Add contract impact analysis in GitHub : Detailed instructions on adding contracts for impact analysis in GitHub."}
{"id":9078,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"Impact analysis helps you identify how modifications to your data contracts might impact downstream processes, data quality, and overall business operations. This can help you analyze proposed changes and mitigate potential risks before implementation."}
{"id":9079,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"If you have ever changed a data contract only to find out later that it broke a downstream table or dashboard, Atlan provides a GitHub Action to help you out. This action places Atlan's impact analysis right into your pull request."}
{"id":9080,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"So, you can view the potential downstream impact of your changes before merging the pull request. Before running the action, you will need to create an Atlan API token."}
{"id":9081,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"You will also need to assign a persona to the API token and add a metadata policy that provides the requisite permissions on assets for the Atlan action to work. For example, you can add the following permissions: Asset, such as a table - Read only Any downstream connections, such as Microsoft Power BI - Read only Asset, such as a table - Read only Any downstream connections, such as Microsoft Power BI - Read only You will need to configure the default GITHUB_TOKEN permissions."}
{"id":9082,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"Grant Read and write permissions to the GITHUB_TOKEN in your repository to allow the atlan-action to seamlessly add or update comments on pull requests. Refer to GitHub documentation to learn more."}
{"id":9083,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"Configure the action To set up the Atlan action in GitHub: Create repository secrets in your repository: ATLAN_INSTANCE_URL with the URL of your Atlan instance. ATLAN_API_TOKEN with the value of the API token."}
{"id":9084,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"ATLAN_INSTANCE_URL with the URL of your Atlan instance. ATLAN_API_TOKEN with the value of the API token."}
{"id":9085,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"Add the GitHub Action to your workflow: Create a workflow file in your repository - .github\/workflows\/atlan-action.yml. Add the following code to your workflow file: name : Atlan action on : pull_request : types : [ opened , edited , synchronize , reopened , closed ] jobs : get-downstream-impact : name : Get Downstream Assets runs-on : ubuntu - latest steps : - name : Checkout uses : actions\/checkout@v4 - name : Run Action uses : atlanhq\/atlan - action@v1 with : GITHUB_TOKEN : $ { { secrets.GITHUB_TOKEN } } ATLAN_INSTANCE_URL : $ { { secrets.ATLAN_INSTANCE_URL } } ATLAN_API_TOKEN : $ { { secrets.ATLAN_API_TOKEN } } ATLAN_CONFIG : .atlan\/config.yaml Create a workflow file in your repository - .github\/workflows\/atlan-action.yml."}
{"id":9086,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"Create a workflow file in your repository - .github\/workflows\/atlan-action.yml. Add the following code to your workflow file: name : Atlan action on : pull_request : types : [ opened , edited , synchronize , reopened , closed ] jobs : get-downstream-impact : name : Get Downstream Assets runs-on : ubuntu - latest steps : - name : Checkout uses : actions\/checkout@v4 - name : Run Action uses : atlanhq\/atlan - action@v1 with : GITHUB_TOKEN : $ { { secrets.GITHUB_TOKEN } } ATLAN_INSTANCE_URL : $ { { secrets.ATLAN_INSTANCE_URL } } ATLAN_API_TOKEN : $ { { secrets.ATLAN_API_TOKEN } } ATLAN_CONFIG : .atlan\/config.yaml Add the following code to your workflow file: Test the action After you've completed the configuration above, create a pull request with a changed Atlan data contract file to test the action."}
{"id":9087,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"You should see the Atlan GitHub action running and then adding comments in your pull request: The GitHub workflow will add and update a single comment for every file change. The impacted assets in the comment will be displayed in a collapsible section and grouped by source and asset type."}
{"id":9088,"url":"https:\/\/docs.atlan.com\/product\/capabilities\/governance\/contracts\/how-tos\/add-contract-impact-analysis-in-github","title":"Add contract impact analysis in GitHub | Atlan Documentation","text":"View impacted assets directly in Atlan. Configure the action Test the action"}
{"id":9089,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack."}
{"id":9090,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts."}
{"id":9091,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks."}
{"id":9092,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing."}
{"id":9093,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management."}
{"id":9094,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows."}
{"id":9095,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization."}
{"id":9096,"url":"https:\/\/docs.atlan.com\/product\/integrations#__docusaurus_skipToContent_fallback","title":"Integrations | Atlan Documentation","text":"Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack."}
{"id":9097,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack."}
{"id":9098,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts."}
{"id":9099,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks."}
{"id":9100,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing."}
{"id":9101,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management."}
{"id":9102,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows."}
{"id":9103,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization."}
{"id":9104,"url":"https:\/\/docs.atlan.com\/product\/integrations#key-concepts","title":"Integrations | Atlan Documentation","text":"Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack."}
{"id":9105,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack."}
{"id":9106,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts."}
{"id":9107,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks."}
{"id":9108,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing."}
{"id":9109,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management."}
{"id":9110,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows."}
{"id":9111,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization."}
{"id":9112,"url":"https:\/\/docs.atlan.com\/product\/integrations#core-offerings","title":"Integrations | Atlan Documentation","text":"Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack."}
{"id":9113,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack."}
{"id":9114,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts."}
{"id":9115,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks."}
{"id":9116,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing."}
{"id":9117,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management."}
{"id":9118,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows."}
{"id":9119,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization."}
{"id":9120,"url":"https:\/\/docs.atlan.com\/product\/integrations#get-started","title":"Integrations | Atlan Documentation","text":"Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack."}
{"id":9121,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9122,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9123,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9124,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9125,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9126,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9127,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9128,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9129,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9130,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9131,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9132,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9133,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9134,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9135,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9136,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9137,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9138,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9139,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9140,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9141,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9142,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9143,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9144,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9145,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9146,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9147,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9148,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9149,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9150,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9151,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9152,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9153,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9154,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9155,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9156,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9157,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9158,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9159,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9160,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9161,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9162,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9163,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9164,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9165,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9166,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9167,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9168,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9169,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9170,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9171,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9172,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9173,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9174,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9175,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9176,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9177,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9178,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9179,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9180,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9181,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9182,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9183,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9184,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9185,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9186,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9187,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9188,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9189,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9190,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9191,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9192,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9193,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9194,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9195,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9196,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9197,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9198,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9199,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9200,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9201,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9202,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9203,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9204,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9205,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9206,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9207,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9208,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9209,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9210,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9211,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9212,"url":"https:\/\/developer.atlan.com\/getting-started\/#an-introductory-walkthrough","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9213,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9214,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9215,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9216,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9217,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9218,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9219,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9220,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9221,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9222,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9223,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9224,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9225,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9226,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9227,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9228,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9229,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9230,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9231,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9232,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9233,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9234,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9235,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9236,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9237,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9238,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9239,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9240,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9241,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9242,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9243,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9244,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9245,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9246,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9247,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9248,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9249,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9250,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9251,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9252,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9253,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9254,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9255,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9256,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9257,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9258,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9259,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9260,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9261,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9262,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9263,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9264,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9265,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9266,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9267,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9268,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9269,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9270,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9271,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9272,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9273,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9274,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9275,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9276,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9277,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9278,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9279,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9280,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9281,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9282,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9283,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9284,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9285,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9286,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9287,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9288,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9289,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9290,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9291,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9292,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9293,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9294,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9295,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9296,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9297,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9298,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9299,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9300,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9301,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9302,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9303,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9304,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A1","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9305,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9306,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9307,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9308,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9309,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9310,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9311,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9312,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9313,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9314,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9315,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9316,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9317,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9318,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9319,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9320,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9321,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9322,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9323,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9324,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9325,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9326,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9327,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9328,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9329,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9330,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9331,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9332,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9333,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9334,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9335,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9336,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9337,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9338,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9339,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9340,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9341,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9342,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9343,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9344,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9345,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9346,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9347,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9348,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9349,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9350,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9351,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9352,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9353,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9354,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9355,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9356,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9357,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9358,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9359,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9360,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9361,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9362,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9363,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9364,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9365,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9366,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9367,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9368,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9369,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9370,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9371,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9372,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9373,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9374,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9375,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9376,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9377,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9378,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9379,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9380,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9381,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9382,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9383,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9384,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9385,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9386,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9387,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9388,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9389,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9390,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9391,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9392,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9393,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9394,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9395,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9396,"url":"https:\/\/developer.atlan.com\/getting-started\/#setting-up","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9397,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency."}
{"id":9398,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes."}
{"id":9399,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"You can include logback as a simple binding mechanism to send any logging information out to your console (standard out), at INFO -level or above. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":9400,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system."}
{"id":9401,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"But it's your choice exactly how you manage the API token and including it for use within the client. (Note that you can also explicity provide only the tenant URL, and the constructor will look for only the API key through an environment variable.) That's it — once these are set you can start using your SDK to make live calls against your Atlan instance!"}
{"id":9402,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":9403,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":9404,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":9405,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":9406,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":9407,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException."}
{"id":9408,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":9409,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":9410,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems."}
{"id":9411,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems."}
{"id":9412,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the ch.qos.logback:logback-classic:1.2.11 logback binding with log4j2 bindings."}
{"id":9413,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException. When there is a 403 response indicating that permission for an operation is not (yet) available."}
{"id":9414,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":9415,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":9416,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"(This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out."}
{"id":9417,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client."}
{"id":9418,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds)."}
{"id":9419,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant."}
{"id":9420,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual."}
{"id":9421,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"You can access the operations for assets directly on the client, under client.assets. These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides."}
{"id":9422,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request."}
{"id":9423,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches."}
{"id":9424,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant."}
{"id":9425,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java. command."}
{"id":9426,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument."}
{"id":9427,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":". and your password as the second argument, as a char[]."}
{"id":9428,"url":"https:\/\/developer.atlan.com\/sdks\/java\/#logging","title":"Java SDK - Developer","text":"(Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":9429,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is currently available on pypi."}
{"id":9430,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"You can use pip to install it as follows: Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token!"}
{"id":9431,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system. But it's your choice exactly how you manage the API token and including it for use within the client."}
{"id":9432,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"In some scenarios, you may not want to expose the entire API token or manage environment variables. Instead, you can provide the GUID of the API token, and the SDK will internally fetch the actual access token."}
{"id":9433,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"When to use this approach: Building apps that use the SDK where token security is a concern When you want to avoid exposing full API tokens in your configuration For containerized applications that need secure token management Before using this approach, ensure your Argo template is configured with CLIENT_ID and CLIENT_SECRET : Create client from token GUID : Use AtlanClient.from_token_guid() to create a client using the GUID of an API token. The SDK will automatically fetch the actual access token using the configured CLIENT_ID and CLIENT_SECRET."}
{"id":9434,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"That's it — once these are set you can start using your SDK to make live calls against your Atlan instance! 🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets."}
{"id":9435,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9436,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9437,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9438,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9439,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above."}
{"id":9440,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError. For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection."}
{"id":9441,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place."}
{"id":9442,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionError will be raised."}
{"id":9443,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":9444,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses logging module of the standard library that can provide a flexible framework for emitting log messages."}
{"id":9445,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). logging.INFO : used to confirm that things are working as expected."}
{"id":9446,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future. logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":9447,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now). You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK)."}
{"id":9448,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"logging.INFO : used to confirm that things are working as expected. logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future."}
{"id":9449,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation. logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now)."}
{"id":9450,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"By default, logs will appear in your console. If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file."}
{"id":9451,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"\/tmp\/pyatlan.json : log file in JSON format. By default, logs will appear in your console."}
{"id":9452,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":9453,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":9454,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 429 response indicating that the request rate limit has been exceeded, and you need to retry after some time."}
{"id":9455,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"When there is a 50x response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":9456,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Before each retry, the SDK will apply a delay using an exponential backoff. (Currently the values for the exponential backoff are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":9457,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"(This is set to 5 by default.) By default, the SDK AtlanClient() has the following timeout settings: read_timeout : 900.0 seconds ( 15 minutes) connect_timeout : 30.0 seconds If you need to override these defaults, you can do so as shown in the example below: Since version 1.0.0, the Python SDK supports connecting to multiple tenants.[^1] When you use the AtlanClient() method you are actually setting a default client. This default client will be used behind-the-scenes for any operations that need information specific to an Atlan tenant."}
{"id":9458,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"When you want to override that default client you can create a new one and use the set_default_client() method to change it: The AtlanClient() method will return a client for the given base URL, creating a new client and setting this new client as the default client. If you want to switch between clients that you have already created, you can use Atlan.set_default_client() to change between them."}
{"id":9459,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume."}
{"id":9460,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. (And since in the majority of use cases you only need access to a single tenant, this means you can most likely just rely on the default client and the fallback behavior.) Pyatlan uses the Requests library which supports proxy configuration via environment variables."}
{"id":9461,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Requests relies on the proxy configuration defined by standard environment variables http_proxy, https_proxy, no_proxy, and all_proxy. Uppercase variants of these variables are also supported."}
{"id":9462,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"You can therefore set them to configure Pyatlan (only set the ones relevant to your needs): To use HTTP Basic Auth with your proxy, use the http:\/\/user:password@host\/ syntax in any of the above configuration entries: Currently, the way this is implemented limits you to either avoiding multiple threads in your Python code (if you need to use multiple clients), or if you want to use multiple threads you should only use a single client. Asynchronous SDK operations To get started, you need to initialize an AsyncAtlanClient : Create an async client using the same configuration pattern as the synchronous client."}
{"id":9463,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"Concurrent operations for improved performance The real power of async comes from running multiple operations concurrently. Instead of waiting for each operation to complete sequentially, you can execute them in parallel and reduce total execution time: Synchronous : Total time = operation₁ + operation₂ +."}
{"id":9464,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#using-environment-variables","title":"Python SDK - Developer","text":"+ operationₙ Asynchronous : Total time = max(operation₁, operation₂,. , operationₙ) When to use async Async is most beneficial when you have: Multiple independent operations that can run concurrently I\/O-heavy workloads like API calls, database queries, or file operations Long-running operations where parallelization provides significant time savings For simple, single operations, the synchronous client may be more straightforward to use."}
{"id":9465,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is currently available on pypi."}
{"id":9466,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"You can use pip to install it as follows: Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token!"}
{"id":9467,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system. But it's your choice exactly how you manage the API token and including it for use within the client."}
{"id":9468,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"In some scenarios, you may not want to expose the entire API token or manage environment variables. Instead, you can provide the GUID of the API token, and the SDK will internally fetch the actual access token."}
{"id":9469,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"When to use this approach: Building apps that use the SDK where token security is a concern When you want to avoid exposing full API tokens in your configuration For containerized applications that need secure token management Before using this approach, ensure your Argo template is configured with CLIENT_ID and CLIENT_SECRET : Create client from token GUID : Use AtlanClient.from_token_guid() to create a client using the GUID of an API token. The SDK will automatically fetch the actual access token using the configured CLIENT_ID and CLIENT_SECRET."}
{"id":9470,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"That's it — once these are set you can start using your SDK to make live calls against your Atlan instance! 🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets."}
{"id":9471,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9472,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9473,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9474,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9475,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above."}
{"id":9476,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError. For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection."}
{"id":9477,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place."}
{"id":9478,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionError will be raised."}
{"id":9479,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":9480,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses logging module of the standard library that can provide a flexible framework for emitting log messages."}
{"id":9481,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). logging.INFO : used to confirm that things are working as expected."}
{"id":9482,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future. logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":9483,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now). You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK)."}
{"id":9484,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"logging.INFO : used to confirm that things are working as expected. logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future."}
{"id":9485,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation. logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now)."}
{"id":9486,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"By default, logs will appear in your console. If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file."}
{"id":9487,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"\/tmp\/pyatlan.json : log file in JSON format. By default, logs will appear in your console."}
{"id":9488,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":9489,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":9490,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 429 response indicating that the request rate limit has been exceeded, and you need to retry after some time."}
{"id":9491,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"When there is a 50x response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":9492,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Before each retry, the SDK will apply a delay using an exponential backoff. (Currently the values for the exponential backoff are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":9493,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"(This is set to 5 by default.) By default, the SDK AtlanClient() has the following timeout settings: read_timeout : 900.0 seconds ( 15 minutes) connect_timeout : 30.0 seconds If you need to override these defaults, you can do so as shown in the example below: Since version 1.0.0, the Python SDK supports connecting to multiple tenants.[^1] When you use the AtlanClient() method you are actually setting a default client. This default client will be used behind-the-scenes for any operations that need information specific to an Atlan tenant."}
{"id":9494,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"When you want to override that default client you can create a new one and use the set_default_client() method to change it: The AtlanClient() method will return a client for the given base URL, creating a new client and setting this new client as the default client. If you want to switch between clients that you have already created, you can use Atlan.set_default_client() to change between them."}
{"id":9495,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume."}
{"id":9496,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. (And since in the majority of use cases you only need access to a single tenant, this means you can most likely just rely on the default client and the fallback behavior.) Pyatlan uses the Requests library which supports proxy configuration via environment variables."}
{"id":9497,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Requests relies on the proxy configuration defined by standard environment variables http_proxy, https_proxy, no_proxy, and all_proxy. Uppercase variants of these variables are also supported."}
{"id":9498,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"You can therefore set them to configure Pyatlan (only set the ones relevant to your needs): To use HTTP Basic Auth with your proxy, use the http:\/\/user:password@host\/ syntax in any of the above configuration entries: Currently, the way this is implemented limits you to either avoiding multiple threads in your Python code (if you need to use multiple clients), or if you want to use multiple threads you should only use a single client. Asynchronous SDK operations To get started, you need to initialize an AsyncAtlanClient : Create an async client using the same configuration pattern as the synchronous client."}
{"id":9499,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"Concurrent operations for improved performance The real power of async comes from running multiple operations concurrently. Instead of waiting for each operation to complete sequentially, you can execute them in parallel and reduce total execution time: Synchronous : Total time = operation₁ + operation₂ +."}
{"id":9500,"url":"https:\/\/developer.atlan.com\/sdks\/python\/#logging","title":"Python SDK - Developer","text":"+ operationₙ Asynchronous : Total time = max(operation₁, operation₂,. , operationₙ) When to use async Async is most beneficial when you have: Multiple independent operations that can run concurrently I\/O-heavy workloads like API calls, database queries, or file operations Long-running operations where parallelization provides significant time savings For simple, single operations, the synchronous client may be more straightforward to use."}
{"id":9501,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Obtain the SDK For Kotlin, you can reuse the existing Java SDK as-is. It is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency."}
{"id":9502,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes."}
{"id":9503,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":9504,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":9505,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":9506,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":9507,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":9508,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":9509,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException."}
{"id":9510,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":9511,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":9512,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems."}
{"id":9513,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems."}
{"id":9514,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the org.slf4j:slf4j-simple:2.0.7 binding with log4j2 bindings."}
{"id":9515,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException. When there is a 403 response indicating that permission for an operation is not (yet) available."}
{"id":9516,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":9517,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":9518,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"(This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out."}
{"id":9519,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client."}
{"id":9520,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds)."}
{"id":9521,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant."}
{"id":9522,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual."}
{"id":9523,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"You can access the operations for assets directly on the client, under client.assets. These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides."}
{"id":9524,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request."}
{"id":9525,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches."}
{"id":9526,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant."}
{"id":9527,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java. command."}
{"id":9528,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument."}
{"id":9529,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":". and your password as the second argument, as a char[]."}
{"id":9530,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/#logging","title":"Kotlin SDK - Developer","text":"(Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":9531,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Obtain the SDK The Go SDK is currently in a pre-release, experimental state. While in this state, we reserve the right to make any changes to it (including breaking changes) without worrying about backwards compatibility, semantic versioning, and so on."}
{"id":9532,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"If you are eager to experiment with it, it is available on GitHub. You can use Go dependencies to install it directly from there."}
{"id":9533,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"We welcome your feedback during the pre-release, but cannot commit to any specific revisions or timelines at this point in time. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":9534,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system."}
{"id":9535,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"But it's your choice exactly how you manage the API token and including it for use within the client. That's it — once these are set you can start using your SDK to make live calls against your Atlan instance!"}
{"id":9536,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":9537,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":9538,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":9539,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":9540,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":9541,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError."}
{"id":9542,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":9543,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":9544,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"In these cases, an ApiConnectionError will be raised. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":9545,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses the slog library internally to provide a flexible framework for emitting log messages."}
{"id":9546,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using .SetLogger() on the context object with various logging levels: \"debug\" : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). \"info\" : used to confirm that things are working as expected."}
{"id":9547,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"\"warn\" : used as an indication that something unexpected happened, or as a warning of some problem in the near future. \"error\" : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":9548,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"You can enable logging by using .SetLogger() on the context object with various logging levels: \"debug\" : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). \"info\" : used to confirm that things are working as expected."}
{"id":9549,"url":"https:\/\/developer.atlan.com\/sdks\/go\/#logging","title":"Go SDK - Developer","text":"\"warn\" : used as an indication that something unexpected happened, or as a warning of some problem in the near future. \"error\" : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":9550,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9551,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9552,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9553,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9554,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9555,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9556,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9557,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9558,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9559,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9560,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9561,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9562,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9563,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9564,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9565,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9566,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9567,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9568,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9569,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9570,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9571,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9572,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9573,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9574,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9575,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9576,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9577,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9578,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9579,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9580,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9581,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9582,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9583,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9584,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9585,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9586,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9587,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9588,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9589,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9590,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9591,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9592,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9593,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9594,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9595,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9596,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9597,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9598,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9599,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9600,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9601,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9602,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9603,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9604,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9605,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9606,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9607,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9608,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9609,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9610,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9611,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9612,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9613,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9614,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9615,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9616,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9617,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9618,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9619,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9620,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9621,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9622,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9623,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9624,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9625,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9626,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9627,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9628,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9629,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9630,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9631,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9632,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9633,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9634,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9635,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9636,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9637,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9638,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9639,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9640,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9641,"url":"https:\/\/developer.atlan.com\/getting-started\/#retrieving-metadata","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9642,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9643,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9644,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9645,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9646,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9647,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9648,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9649,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9650,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9651,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9652,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9653,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9654,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9655,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9656,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9657,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9658,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9659,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9660,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9661,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9662,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9663,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9664,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9665,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9666,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9667,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9668,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9669,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9670,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9671,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9672,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9673,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9674,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9675,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9676,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9677,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9678,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9679,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9680,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9681,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9682,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9683,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9684,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9685,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9686,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9687,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9688,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9689,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9690,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9691,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9692,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9693,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9694,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9695,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9696,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9697,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9698,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9699,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9700,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9701,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9702,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9703,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9704,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9705,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9706,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9707,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9708,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9709,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9710,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9711,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9712,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9713,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9714,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9715,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9716,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9717,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9718,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9719,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9720,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9721,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9722,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9723,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9724,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9725,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9726,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9727,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9728,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9729,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9730,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9731,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9732,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9733,"url":"https:\/\/developer.atlan.com\/getting-started\/#what-is-an-asset","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9734,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9735,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9736,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9737,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9738,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9739,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9740,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9741,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9742,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9743,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9744,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9745,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9746,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9747,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9748,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9749,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9750,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9751,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9752,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9753,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9754,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9755,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9756,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9757,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9758,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9759,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9760,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9761,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9762,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9763,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9764,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9765,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9766,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9767,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9768,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9769,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9770,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9771,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9772,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9773,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9774,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9775,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9776,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9777,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9778,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9779,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9780,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9781,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9782,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9783,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9784,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9785,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9786,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9787,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9788,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9789,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9790,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9791,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9792,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9793,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9794,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9795,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9796,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9797,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9798,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9799,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9800,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9801,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9802,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9803,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9804,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9805,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9806,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9807,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9808,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9809,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9810,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9811,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9812,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9813,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9814,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9815,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9816,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9817,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9818,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9819,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9820,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9821,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9822,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9823,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9824,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9825,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-know-the-asset","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9826,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Strictly speaking, no, you do not. And in fact if you ultimately intend to update an asset you should trim it down to only what you intend to change and not send a complete asset."}
{"id":9827,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"See Updating an asset for more details. Retrieving an asset uses a slightly different pattern from the other operations."}
{"id":9828,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"For this you can use static methods provided by the Asset class: To retrieve an asset by its GUID: If no exception is thrown, the returned object will be non-null and of the type requested. Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":9829,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Compile-time type checking This operation will type-check the asset you are retrieving is of the type requested. If it is not, you will receive a NotFoundException , even if the GUID represents some other asset."}
{"id":9830,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"client.asset.get_by_guid() method takes following parameters: guid : specify the (GUID) of the asset to retrieve. asset_type ( optional ): specify the type of asset to retrieve."}
{"id":9831,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Defaults to Asset. If no exception is thrown, the returned object will be non-null and of the type requested."}
{"id":9832,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"min_ext_info ( optional ): minimizes additional information when set to True. Defaults to False ignore_relationships ( optional ): specify whether to include relationships ( False ) or exclude them ( True )."}
{"id":9833,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Defaults to True attributes ( optional ): defines the list of attributes to retrieve for the asset. Accepts either a list of strings or a list of AtlanField."}
{"id":9834,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"related_attributes ( optional ): defines the list of relationship attributes to retrieve for the asset. Accepts either a list of strings or a list of AtlanField."}
{"id":9835,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Attributes and Related attributes Run-time type checking This operation will type-check the asset you are retrieving is of the type requested. If it is not, you will receive a NotFoundException , even if the GUID represents some other asset."}
{"id":9836,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"If no exception is thrown, the returned object will be non-null and of the type requested. Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":9837,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Compile-time type checking This operation will type-check the asset you are retrieving is of the type requested. If it is not, you will receive a NotFoundException , even if the GUID represents some other asset."}
{"id":9838,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"In the case of retrieving an asset, all necessary information is included in the URL of the request. There is no payload for the body of the request."}
{"id":9839,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"By GUID (runtime typing) To retrieve an asset by GUID, but only resolve the type at runtime: Retrieve the asset by its GUID. Since GUIDs are globally unique, you do not need to specify a type."}
{"id":9840,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"(And this is why the operation returns a generic Asset , since the SDK can only determine the type at runtime, once it has a response back from Atlan.) Since the operation returns a generic Asset , you need to check and cast it to a more specific type if you want to access the more specific attributes of that type. client.asset.get_by_guid() method takes following parameters: guid : specify the (GUID) of the asset to retrieve."}
{"id":9841,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"asset_type ( optional ): specify the type of asset to retrieve. Defaults to Asset."}
{"id":9842,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"If no exception is thrown, the returned object will be non-null and of the type requested. min_ext_info ( optional ): minimizes additional information when set to True."}
{"id":9843,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Defaults to False ignore_relationships ( optional ): specify whether to include relationships ( False ) or exclude them ( True ). Defaults to True attributes ( optional ): defines the list of attributes to retrieve for the asset."}
{"id":9844,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Accepts either a list of strings or a list of AtlanField. related_attributes ( optional ): defines the list of relationship attributes to retrieve for the asset."}
{"id":9845,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Accepts either a list of strings or a list of AtlanField. Attributes and Related attributes Since the operation returns a generic Asset , you need to use isinstance() to cast it to a more specific type in the block if you want an IDE to provide more specific type hints."}
{"id":9846,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Since the operation returns a generic Asset , you need to use isinstance() to cast it to a more specific type in the block if you want an IDE to provide more specific type hints. Retrieve the asset by its GUID."}
{"id":9847,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Since GUIDs are globally unique, you do not need to specify a type. (And this is why the operation returns a generic Asset , since the SDK can only determine the type at runtime, once it has a response back from Atlan.) Since the operation returns a generic Asset , you need to check and cast it to a more specific type if you want to access the more specific attributes of that type."}
{"id":9848,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Does not apply to a raw API request There is no concept of typing in a raw API request — all responses to the raw API will simply be JSON objects. To retrieve an asset by its qualifiedName : If no exception is thrown, the returned object will be non-null and of the type requested."}
{"id":9849,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant. Qualified name, not name For most objects, you can probably build-up the qualifiedName in your code directly."}
{"id":9850,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant. Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example)."}
{"id":9851,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"To find this portion, see Find connections. For most objects, you can probably build-up the qualifiedName in your code directly."}
{"id":9852,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant. Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example)."}
{"id":9853,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"To find this portion, see Find connections. client.asset.get_by_qualified_name() method takes following parameters: qualified_name : specify the qualified name of the asset to retrieve."}
{"id":9854,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"asset_type : specify the type of asset to retrieve. If no exception is thrown, the returned object will be non-null and of the type requested."}
{"id":9855,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"min_ext_info ( optional ): minimizes additional information when set to True. Defaults to False ignore_relationships ( optional ): specify whether to include relationships ( False ) or exclude them ( True )."}
{"id":9856,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Defaults to True attributes ( optional ): defines the list of attributes to retrieve for the asset. Accepts either a list of strings or a list of AtlanField."}
{"id":9857,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"related_attributes ( optional ): defines the list of relationship attributes to retrieve for the asset. Accepts either a list of strings or a list of AtlanField."}
{"id":9858,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Attributes and Related attributes For most objects, you can probably build-up the qualified_name in your code directly. Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example)."}
{"id":9859,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"To find this portion, see Find connections. For most objects, you can probably build-up the qualified_name in your code directly."}
{"id":9860,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example). To find this portion, see Find connections."}
{"id":9861,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"If no exception is thrown, the returned object will be non-null and of the type requested. Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":9862,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Qualified name, not name For most objects, you can probably build-up the qualifiedName in your code directly. Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":9863,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example). To find this portion, see Find connections."}
{"id":9864,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"For most objects, you can probably build-up the qualifiedName in your code directly. Because this operation will read the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":9865,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Finding the connection portion The one exception is likely to be the connection portion of the name ( default\/snowflake\/1657037873 in this example). To find this portion, see Find connections."}
{"id":9866,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"In the case of retrieving an asset, all necessary information is included in the URL of the request. There is no payload for the body of the request."}
{"id":9867,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"URL encoding may be needed Note that depending on the qualifiedName, you may need to URL-encode its value before sending. This is to replace any parts of the name that could be misinterpreted as actual URL components (like \/ or spaces)."}
{"id":9868,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"In the case of retrieving an asset, all necessary information is included in the URL of the request. There is no payload for the body of the request."}
{"id":9869,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"URL encoding may be needed Note that depending on the qualifiedName, you may need to URL-encode its value before sending. This is to replace any parts of the name that could be misinterpreted as actual URL components (like \/ or spaces)."}
{"id":9870,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Full vs minimal assets The examples above illustrate how to retrieve: an asset with all of its relationships (a complete asset). an asset without any of its relationships (a minimal asset)."}
{"id":9871,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"You can also retrieve the opposite by explicitly requesting it: Retrieve the full asset, with all of its relationships, by its GUID. The last (optional) parameter being true indicates you want to retrieve the asset with all its relationships (a \"full\" asset)."}
{"id":9872,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Similar variations exist on every asset as well as on the dynamically-typed Asset static methods. Optionally, you can provide the asset type: If no exception is thrown, the returned object will be non-null and of the type requested."}
{"id":9873,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"If no exception is thrown, the returned object will be non-null and of the type requested. Retrieve the full asset, with all of its relationships, by its GUID."}
{"id":9874,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"The last (optional) parameter being true indicates you want to retrieve the asset with all its relationships (a \"full\" asset). Similar variations exist on every asset as well as on the dynamically-typed Asset static methods."}
{"id":9875,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"In the case of retrieving an asset, all necessary information is included in the URL of the request. Retrieving a minimal asset is a matter of setting the query parameters ignoreRelationships and minExtInfo to true."}
{"id":9876,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/read\/","title":"Retrieving assets - Developer","text":"Retrieve minimal assets where possible You should retrieve minimal assets for better performance in cases where you do not need all of the relationships of the asset. Keep in mind that although the relationships will not be visible in the object after retrieving a minimal asset, this does not mean that there are no relationships on that asset (in Atlan)."}
{"id":9877,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9878,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9879,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9880,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9881,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9882,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9883,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9884,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9885,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9886,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9887,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9888,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9889,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9890,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9891,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9892,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9893,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9894,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9895,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9896,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9897,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9898,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9899,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9900,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9901,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9902,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9903,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9904,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9905,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9906,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9907,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9908,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9909,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9910,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":9911,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":9912,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":9913,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":9914,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":9915,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":9916,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":9917,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":9918,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":9919,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":9920,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":9921,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":9922,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":9923,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":9924,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":9925,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":9926,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":9927,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":9928,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9929,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9930,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9931,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9932,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":9933,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":9934,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9935,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9936,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":9937,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":9938,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":9939,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":9940,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":9941,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":9942,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":9943,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":9944,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":9945,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":9946,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9947,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9948,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9949,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9950,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9951,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9952,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9953,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9954,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9955,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":9956,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":9957,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":9958,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":9959,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":9960,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":9961,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":9962,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":9963,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":9964,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":9965,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":9966,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":9967,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":9968,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":9969,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":9970,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":9971,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9972,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":9973,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9974,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9975,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9976,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9977,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":9978,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9979,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":9980,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":9981,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":9982,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":9983,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":9984,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9985,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":9986,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":9987,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":9988,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":9989,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":9990,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":9991,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":9992,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":9993,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":9994,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":9995,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":9996,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":9997,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":9998,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":9999,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10000,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10001,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10002,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10003,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10004,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10005,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10006,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10007,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10008,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10009,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10010,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10011,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10012,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10013,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10014,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10015,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10016,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10017,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10018,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10019,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10020,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10021,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10022,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10023,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10024,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10025,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10026,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10027,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10028,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10029,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10030,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10031,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10032,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10033,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10034,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10035,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10036,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10037,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10038,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10039,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10040,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10041,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10042,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10043,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10044,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10045,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10046,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10047,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10048,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10049,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10050,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10051,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10052,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10053,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10054,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10055,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10056,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10057,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10058,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10059,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10060,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10061,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"Full model reference Our SDKs are strongly-typed to represent every type 's attributes and relationships directly. This reduces the burden on you to understand how these are named, capitalized, spelled, or even the type of data they possess — since all of those details are encoded directly into the models the SDKs use."}
{"id":10062,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"Here you will find a complete reference of all of these attributes and relationships. In certain rare cases, our strongly-typed model might not (yet) include a property you need to access."}
{"id":10063,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"In these cases, you can always directly access the raw properties returned in every API response. For example, when retrieving workflow credentials the response object does not include extra details that are connector-specific."}
{"id":10064,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"However, you can still access these extra properties: Use the getRawJsonObject() accessor to access the entire (raw) response object from the API. For this credential example getRawJsonObject() looks like: { \"id\" : \"e3d74922-e2e8-4dbe-a7ed-3937e5153a51\" , \"version\" : \"super-king-1886\" , \"isActive\" : true , \"createdAt\" : 1719240600149 , \"updatedAt\" : 1719240600149 , \"createdBy\" : \"service-account-apikey-5aca94d3-f0a3-4af5-b074-567e209c1b75\" , \"tenantId\" : \"default\" , \"name\" : \"default-snowflake-1719240599-0\" , \"description\" : null , \"connectorConfigName\" : \"atlan-connectors-snowflake\" , \"connector\" : \"snowflake\" , \"connectorType\" : \"jdbc\" , \"authType\" : \"basic\" , \"host\" : \"abc123.snowflakecomputing.com\" , \"port\" : 443 , \"metadata\" : null , \"level\" : null , \"connection\" : null , \"username\" : \"atlan-user\" , \"extra\" : { \"role\" : \"ADMIN\" , \"warehouse\" : \"DEV\" } } Use the getRawJsonObject() accessor to access the entire (raw) response object from the API."}
{"id":10065,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"For this credential example getRawJsonObject() looks like: Use __atlan_extra__ dict field for any response model to access extra API response properties. For this credential example __atlan_extra__ looks like: { \"username\" : \"atlan-user\" , \"extra\" : { \"role\" : \"ADMIN\" , \"warehouse\" : \"DEV\" } } Use __atlan_extra__ dict field for any response model to access extra API response properties."}
{"id":10066,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"For this credential example __atlan_extra__ looks like: Use the rawJsonObject field to access the entire (raw) response object from the API. For this credential example rawJsonObject looks like: { \"id\" : \"e3d74922-e2e8-4dbe-a7ed-3937e5153a51\" , \"version\" : \"super-king-1886\" , \"isActive\" : true , \"createdAt\" : 1719240600149 , \"updatedAt\" : 1719240600149 , \"createdBy\" : \"service-account-apikey-5aca94d3-f0a3-4af5-b074-567e209c1b75\" , \"tenantId\" : \"default\" , \"name\" : \"default-snowflake-1719240599-0\" , \"description\" : null , \"connectorConfigName\" : \"atlan-connectors-snowflake\" , \"connector\" : \"snowflake\" , \"connectorType\" : \"jdbc\" , \"authType\" : \"basic\" , \"host\" : \"abc123.snowflakecomputing.com\" , \"port\" : 443 , \"metadata\" : null , \"level\" : null , \"connection\" : null , \"username\" : \"atlan-user\" , \"extra\" : { \"role\" : \"ADMIN\" , \"warehouse\" : \"DEV\" } } Use the rawJsonObject field to access the entire (raw) response object from the API."}
{"id":10067,"url":"https:\/\/developer.atlan.com\/models\/","title":"Full model reference - Developer","text":"For this credential example rawJsonObject looks like: These are all the metadata assets currently available through Atlan's SDKs:"}
{"id":10068,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10069,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10070,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10071,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10072,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10073,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10074,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10075,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10076,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10077,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10078,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10079,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10080,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10081,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10082,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10083,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10084,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10085,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10086,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10087,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10088,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10089,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10090,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10091,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10092,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10093,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10094,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10095,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10096,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10097,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10098,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10099,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10100,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10101,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10102,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10103,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10104,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10105,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10106,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10107,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10108,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10109,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10110,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10111,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10112,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10113,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10114,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10115,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10116,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10117,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10118,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10119,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10120,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10121,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10122,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10123,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10124,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10125,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10126,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10127,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10128,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10129,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10130,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10131,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10132,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10133,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10134,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10135,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10136,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10137,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10138,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10139,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10140,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10141,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10142,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10143,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10144,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10145,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10146,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10147,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10148,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10149,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10150,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10151,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10152,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10153,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10154,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10155,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10156,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10157,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10158,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10159,"url":"https:\/\/developer.atlan.com\/getting-started\/#when-you-need-to-find-it-first","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10160,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Build the query 0.1.0 1.0.0 2.0.0 You can start building a query across all assets using the select() method on the assets member of any client. You can chain as many conditions as you want: where() is mandatory inclusion whereNot() is mandatory exclusion whereSome() for conditions where some of them must match You can start building a query across all assets using the select() method on the assets member of any client."}
{"id":10161,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can chain as many conditions as you want: where() is mandatory inclusion whereNot() is mandatory exclusion whereSome() for conditions where some of them must match This helper provides a query that ensures results are active (not archived) assets. Equivalent Elastic query Query beActive = TermQuery."}
{"id":10162,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"of ( m -> m. field ( \"__state\" )."}
{"id":10163,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"value ( AtlanStatus. ACTIVE."}
{"id":10164,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"getValue ())). _toQuery (); This helper provides a query that ensures results are active (not archived) assets."}
{"id":10165,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This helper provides a query that ensures results are active (not archived) assets. You can start building a query across all assets using the select() method on the assets member of any client."}
{"id":10166,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can chain as many mandatory ( where() ) conditions, mandatory exclusion ( whereNot() ) conditions, and set of conditions some of which must match ( whereSome() ) as you want. This helper provides a query that ensures results are active (not archived) assets."}
{"id":10167,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Equivalent Elastic query val beActive = TermQuery. of ( m -> m."}
{"id":10168,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"field ( \"__state\" ). value ( AtlanStatus."}
{"id":10169,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"ACTIVE. getValue ()))."}
{"id":10170,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"_toQuery () This helper provides a query that ensures results are active (not archived) assets. This helper provides a query that ensures results are active (not archived) assets."}
{"id":10171,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"A bool query combines together multiple conditions. A filter clause exactly matches all of the conditions, without scoring (so can be slightly faster than other scoring-based combination mechanisms)."}
{"id":10172,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Term queries are generally used to exactly match values. The __state field will match the status of an asset in Atlan."}
{"id":10173,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"So in this example you will only match assets that are currently ACTIVE (not archived or soft-deleted). You will also only match assets that are of a specific type, since __typeName.keyword will match the type of asset."}
{"id":10174,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Note these names do not exactly match attribute names Build the request 0.0.17 1.0.0 1.1.0 The number of results to include (per page). You can chain as many attributes to include on each related asset to each result."}
{"id":10175,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Since we are returning anchor relationships, this will ensure that the certificateStatus of those related glossaries is also included in each result. The number of results to include (per page)."}
{"id":10176,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can chain as many attributes to include on each related asset to each result. Since we are returning anchor relationships, this will ensure that the certificate_status of those related glossaries is also included in each result."}
{"id":10177,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The number of results to include (per page). You can chain as many attributes to include on each related asset to each result."}
{"id":10178,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Since we are returning anchor relationships, this will ensure that the certificateStatus of those related glossaries is also included in each result. The number of results to include (per page)."}
{"id":10179,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can chain as many attributes to include on each related asset to each result. Since we are returning anchor relationships, this will ensure that the certificateStatus of those related glossaries is also included in each result."}
{"id":10180,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"A query should always be defined within the dsl portion of the request. In addition to the query, you can specify from and size parameters for pagination."}
{"id":10181,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The query itself should be provided within the query portion of the dsl. Here you would use the query body provided in the earlier step."}
{"id":10182,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You must set track_total_hits to true if you want an exact count of the number of results (in particular for pagination). The list of attributes to include on each relationship that is included in each result."}
{"id":10183,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Since we are returning anchor relationships, this will ensure that the certificateStatus of those related glossaries is also included in each result. (Optional) Build request directly from JSON A query should always be defined within the dsl portion of the request."}
{"id":10184,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"In addition to the query, you can specify from and size parameters for pagination. The query itself should be provided within the query portion of the dsl."}
{"id":10185,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Here you would use the query body provided in the earlier step. You must set track_total_hits to true if you want an exact count of the number of results (in particular for pagination)."}
{"id":10186,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The list of attributes to include on each relationship that is included in each result. Since we are returning anchor relationships, this will ensure that the certificateStatus of those related glossaries is also included in each result."}
{"id":10187,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"0.1.0 1.4.0 4.0.0 The getApproximateCount() method gives the total number of results overall (not restricted by page). The count property gives the total number of results overall (not restricted by page)."}
{"id":10188,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The .approximateCount member gives the total number of results overall (not restricted by page). The ApproximateCount property gives the total number of results overall (not restricted by page)."}
{"id":10189,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"If an error occurs during iteration (e.g: a failed API request), it is sent to the errIter channel. The iteration stops, and you can handle the error accordingly."}
{"id":10190,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Implicit in the previous step Iterate through results One page of results 0.1.0 1.0.0 1.1.0 To iterate through one page of results, loop through the list of assets: The page of results itself can be accessed through the getAssets() method on the response. You can then iterate through these results from a single page."}
{"id":10191,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can iterate through the results from a single page. The page of results itself can be accessed through the .assets member on the response."}
{"id":10192,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can then iterate through these results from a single page. You can iterate through the results from a single page."}
{"id":10193,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can filter the asset based on the typename and perform operations on the asset. Each object in entities is a matching asset Each item in the entities array of the response will give details about a matching asset."}
{"id":10194,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Multiple pages of results 0.0.17 1.0.0 1.1.0 To iterate through multiple pages of results: You can simply iterate over the reponse itself. This will lazily load and loop through each page of results until the loop finishes or you break out of it."}
{"id":10195,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"(You could also use response.forEach() , which uses the same iteratable-based implementation behind-the-scenes.) Alternatively, you can also stream the results direct from the response. This will also lazily load and loop through each page of results."}
{"id":10196,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Can be chained without creating a request in-between You can actually chain the stream() method directly onto the end of your query and request construction, without creating a request or response object in-between. Alternatively, you can also stream the results direct from the response."}
{"id":10197,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This will also lazily load and loop through each page of results. Can be chained without creating a request in-between You can actually chain the stream() method directly onto the end of your query and request construction, without creating a request or response object in-between."}
{"id":10198,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"With streaming, you can apply your own limits to the maximum number of results you want to process. Independent of page size Note that this is independent of page size."}
{"id":10199,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You could page through results 50 at a time, but only process a maximum of 100 total results this way. Since the results are lazily-loaded when streaming, only the first two pages of results would be retrieved in such a scenario."}
{"id":10200,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"With streaming, you can apply your own limits to the maximum number of results you want to process. Independent of page size Note that this is independent of page size."}
{"id":10201,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You could page through results 50 at a time, but only process a maximum of 100 total results this way. Since the results are lazily-loaded when streaming, only the first two pages of results would be retrieved in such a scenario."}
{"id":10202,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can also apply your own logical filters to the results. Push-down as much as you can to the query You should of course push-down as many of the filters as you can to the query itself, but if you have a particular complex check to apply that cannot be encoded in the query this can be a useful secondary filter over the results."}
{"id":10203,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"You can also apply your own logical filters to the results. Push-down as much as you can to the query You should of course push-down as many of the filters as you can to the query itself, but if you have a particular complex check to apply that cannot be encoded in the query this can be a useful secondary filter over the results."}
{"id":10204,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The forEach() on the resulting stream will then apply whatever actions you want with the results that come through. The forEach() on the resulting stream will then apply whatever actions you want with the results that come through."}
{"id":10205,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The current_page() method returns a list of the assets for the current page. If there are none then an empty list will be returned."}
{"id":10206,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Iterate through the assets in the current page. The next_pages() method retrieves the next page of results and return True if more assets are available and False if they are not."}
{"id":10207,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Break out of the While loop if no more assets are available. This will iterate through all the results without the need to be concerned with pages."}
{"id":10208,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Iterating over results produces a Generator This means that results are retrieved from the backend a page at time. This also means that you can only iterate over the results once."}
{"id":10209,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This will iterate through all the results without the need to be concerned with pages. Iterating over results produces a Generator This means that results are retrieved from the backend a page at time."}
{"id":10210,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This also means that you can only iterate over the results once. You can simply iterate over the reponse itself."}
{"id":10211,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This will lazily load and loop through each page of results until the loop finishes or you break out of it. (You could also use response.forEach{ } , which uses the same iteratable-based implementation behind-the-scenes.) Alternatively, you can also stream the results direct from the response."}
{"id":10212,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"This will also lazily load and loop through each page of results. Can be chained without creating a request in-between You can actually chain the stream() method directly onto the end of your query and request construction, without creating a request or response object in-between."}
{"id":10213,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Alternatively, you can also stream the results direct from the response. This will also lazily load and loop through each page of results."}
{"id":10214,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Can be chained without creating a request in-between You can actually chain the stream() method directly onto the end of your query and request construction, without creating a request or response object in-between. With streaming, you can apply your own limits to the maximum number of results you want to process."}
{"id":10215,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Independent of page size Note that this is independent of page size. You could page through results 50 at a time, but only process a maximum of 100 total results this way."}
{"id":10216,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Since the results are lazily-loaded when streaming, only the first two pages of results would be retrieved in such a scenario. With streaming, you can apply your own limits to the maximum number of results you want to process."}
{"id":10217,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Independent of page size Note that this is independent of page size. You could page through results 50 at a time, but only process a maximum of 100 total results this way."}
{"id":10218,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Since the results are lazily-loaded when streaming, only the first two pages of results would be retrieved in such a scenario. You can also apply your own logical filters to the results."}
{"id":10219,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Push-down as much as you can to the query You should of course push-down as many of the filters as you can to the query itself, but if you have a particular complex check to apply that cannot be encoded in the query this can be a useful secondary filter over the results. You can also apply your own logical filters to the results."}
{"id":10220,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Push-down as much as you can to the query You should of course push-down as many of the filters as you can to the query itself, but if you have a particular complex check to apply that cannot be encoded in the query this can be a useful secondary filter over the results. The forEach{ } on the resulting stream will then apply whatever actions you want with the results that come through."}
{"id":10221,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The forEach{ } on the resulting stream will then apply whatever actions you want with the results that come through. If an error occurs during iteration (e.g: a failed API request), it is sent to the errIter channel."}
{"id":10222,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"The iteration stops, and you can handle the error accordingly. Use this query string from the response to start building a new query using the same logic."}
{"id":10223,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Add the page size to the from parameter embedded in that query string, to give the starting point for the next page of results. Re-include any attributes or relationAttributes from the query string into the new query."}
{"id":10224,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/search\/","title":"Searching for assets - Developer","text":"Send this new query to retrieve the next page of results."}
{"id":10225,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10226,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10227,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10228,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10229,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10230,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10231,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10232,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10233,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10234,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10235,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10236,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10237,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10238,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10239,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10240,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10241,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10242,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10243,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10244,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10245,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10246,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10247,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10248,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10249,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10250,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10251,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10252,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10253,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10254,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10255,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10256,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10257,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10258,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10259,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10260,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10261,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10262,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10263,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10264,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10265,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10266,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10267,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10268,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10269,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10270,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10271,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10272,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10273,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10274,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10275,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10276,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10277,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10278,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10279,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10280,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10281,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10282,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10283,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10284,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10285,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10286,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10287,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10288,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10289,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10290,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10291,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10292,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10293,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10294,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10295,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10296,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10297,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10298,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10299,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10300,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10301,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10302,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10303,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10304,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10305,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10306,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10307,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10308,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10309,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10310,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10311,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10312,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10313,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10314,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10315,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10316,"url":"https:\/\/developer.atlan.com\/getting-started\/#updating-metadata","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10317,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Updating an asset All objects in the SDK that you can update within Atlan implement the builder pattern. This allows you to progressively build-up the object you want to update."}
{"id":10318,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"In addition, each object provides a method that takes the minimal set of required fields to update that asset , when it already exists in Atlan. Include only your intended changes, nothing more When enriching an asset in Atlan, you only need to specify the information you want to change."}
{"id":10319,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Any information you do not include in your update will be left untouched on the asset in Atlan. This way you do not need to: try to reproduce the complete asset in your request to do targeted updates to specific attributes worry about other changes that may be made to the asset in parallel to the changes you will be making to the asset Build minimal object needed For example, to update a glossary term we need to provide the qualifiedName and name of the term, and the GUID of the glossary in which it exists: The name of the existing term."}
{"id":10320,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"This must match exactly (case-sensitive). The GUID of the glossary in which the term exists."}
{"id":10321,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The name of the existing term. This must match exactly (case-sensitive)."}
{"id":10322,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The GUID of the glossary in which the term exists. The name of the existing term."}
{"id":10323,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"This must match exactly (case-sensitive). The GUID of the glossary in which the term exists."}
{"id":10324,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Implicit in the API calls below There is nothing specific to do for this step when using the raw APIs — constructing the object is simply what you place in the payload of the API calls in the steps below. Since you should only include your intended changes, and nothing more, the SDKs provide a convenience method to reduce an asset down to its minimal properties."}
{"id":10325,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You should use this method to trim an object in your code down to a starting point for the changes you want to make to that asset: Assuming you have an existing asset in a variable called existing , you can call trimToRequired() to reduce it to a builder with the minimal properties needed to update that asset. Assuming you have an existing asset in a variable called existing , you can call trim_to_required() to reduce it to an object with the minimal properties needed to update that asset."}
{"id":10326,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Enrich before updating The term object now has the minimal required information for Atlan to update it. Without any additional enrichment, though, there isn't really anything to update."}
{"id":10327,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"So first you should enrich the object: We'll create an object we can take actions on from this updater. In this example, we're adding a certificate to the object."}
{"id":10328,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Note that you can chain any number of enrichments together. Here we are also adding an announcement to the asset."}
{"id":10329,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To persist the enrichment back to the object, we must build() the builder. Assign the result back Remember to assign the result of the build() operation back to your original object."}
{"id":10330,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Otherwise the result is not persisted back into any variable! (In this case we're assigning to the term variable back on line 5.) To persist the enrichment back to the object, we must build() the builder."}
{"id":10331,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Assign the result back Remember to assign the result of the build() operation back to your original object. Otherwise the result is not persisted back into any variable!"}
{"id":10332,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(In this case we're assigning to the term variable back on line 5.) In this example, we're adding a certificate to the object. In this example, we're adding an announcement to the object."}
{"id":10333,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"We'll create an object we can take actions on from this updater. In this example, we're adding a certificate to the object."}
{"id":10334,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Note that you can chain any number of enrichments together. Here we are also adding an announcement to the asset."}
{"id":10335,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To persist the enrichment back to the object, we must build() the builder. Assign the result back Remember to assign the result of the build() operation back to your original object."}
{"id":10336,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Otherwise the result is not persisted back into any variable! (In this case we're assigning to the term variable back on line 5.) To persist the enrichment back to the object, we must build() the builder."}
{"id":10337,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Assign the result back Remember to assign the result of the build() operation back to your original object. Otherwise the result is not persisted back into any variable!"}
{"id":10338,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(In this case we're assigning to the term variable back on line 5.) Implicit in the API calls below There is nothing specific to do for this step when using the raw APIs — constructing the object is simply what you place in the payload of the API calls in the steps below. Update the asset from the object You can then actually update the object in Atlan 1 : The save() method will either update an existing asset (if Atlan already has a term with the same name and qualifiedName in the same glossary) or create a new asset (if Atlan does not have a term with the same name in the same glossary)."}
{"id":10339,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant. You can distinguish what was created or updated: getCreatedAssets() lists assets that were created getUpdatedAssets() lists assets that were updated Note that the save() method always returns objects of type Asset , though."}
{"id":10340,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: getCreatedAssets() lists assets that were created getUpdatedAssets() lists assets that were updated Note that the save() method always returns objects of type Asset , though. The Asset class is a superclass of all assets."}
{"id":10341,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned. The Asset class is a superclass of all assets."}
{"id":10342,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned. The updateMergingCM() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualifiedName )."}
{"id":10343,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant. Depending on the update behavior you want, you could also use: updateMergingCM(false) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched updateMergingCM(true) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset updateReplacingCM(false) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched updateReplacingCM(true) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset The updateMergingCM() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualifiedName )."}
{"id":10344,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant. Depending on the update behavior you want, you could also use: updateMergingCM(false) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched updateMergingCM(true) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset updateReplacingCM(false) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched updateReplacingCM(true) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset You can distinguish what was created or updated: getUpdatedAssets() lists assets that were updated Note that the update."}
{"id":10345,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"() methods always returns objects of type Asset , though. You can distinguish what was created or updated: getUpdatedAssets() lists assets that were updated Note that the update."}
{"id":10346,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"() methods always returns objects of type Asset , though. The Asset class is a superclass of all assets."}
{"id":10347,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned. The Asset class is a superclass of all assets."}
{"id":10348,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned. Since the update."}
{"id":10349,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"() methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundException. Since the update."}
{"id":10350,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"() methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundException. The save(term) method will either update an existing asset (if Atlan already has a term with the same name and qualifiedName in the same glossary) or (create a new asset, if Atlan does not have a term with the same name in the same glossary)."}
{"id":10351,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: assets_created(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were created. assets_updated(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were updated."}
{"id":10352,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: assets_created(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were created. assets_updated(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were updated."}
{"id":10353,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"If the returned list is not empty, get the term that was updated. If the returned list is not empty, get the term that was updated."}
{"id":10354,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The update_merging_cm() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualified_name ). Depending on the update behavior you want, you could also use: update_merging_cm(replace_atlan_tags=False) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched update_merging_cm(replace_atlan_tags=True) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset update_replacing_cm(replace_atlan_tags=False) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched update_replacing_cm(replace_atlan_tags=True) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset The update_merging_cm() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualified_name )."}
{"id":10355,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Depending on the update behavior you want, you could also use: update_merging_cm(replace_atlan_tags=False) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched update_merging_cm(replace_atlan_tags=True) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset update_replacing_cm(replace_atlan_tags=False) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched update_replacing_cm(replace_atlan_tags=True) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset You can distinguish what was created or updated: assets_updated(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were updated. You can distinguish what was created or updated: assets_updated(asset_type = AtlasGlossaryType) returns a list assets of the specified type that were updated."}
{"id":10356,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Since the update. () methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundError."}
{"id":10357,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Since the update. () methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundError."}
{"id":10358,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The save() method will either update an existing asset (if Atlan already has a term with the same name and qualifiedName in the same glossary) or create a new asset (if Atlan does not have a term with the same name in the same glossary). Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":10359,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: createdAssets lists assets that were created updatedAssets lists assets that were updated Note that the save() method always returns objects of type Asset , though. You can distinguish what was created or updated: createdAssets lists assets that were created updatedAssets lists assets that were updated Note that the save() method always returns objects of type Asset , though."}
{"id":10360,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10361,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10362,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The updateMergingCM() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualifiedName ). Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":10363,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Depending on the update behavior you want, you could also use: updateMergingCM(false) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched updateMergingCM(true) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset updateReplacingCM(false) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched updateReplacingCM(true) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset The updateMergingCM() method will only update an existing asset (if Atlan already has an asset of the same type with the same name qualifiedName ). Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":10364,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Depending on the update behavior you want, you could also use: updateMergingCM(false) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while leaving any Atlan tags on the existing asset untouched updateMergingCM(true) to only overwrite any custom metadata provided in your update (leaving anything you don't provide untouched on the existing asset), while replacing any Atlan tags on the existing asset updateReplacingCM(false) to overwrite all custom metadata on the existing asset with what you're providing in your update, while leaving any Atlan tags on the existing asset untouched updateReplacingCM(true) to overwrite all custom metadata on the existing asset with what you're providing in your update, while replacing any Atlan tags on the existing asset You can distinguish what was created or updated: getUpdatedAssets() lists assets that were updated Note that the update. () methods always returns objects of type Asset , though."}
{"id":10365,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: getUpdatedAssets() lists assets that were updated Note that the update. () methods always returns objects of type Asset , though."}
{"id":10366,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10367,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10368,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Since the update. () methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundException."}
{"id":10369,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Since the update. () methods strictly update (and never create) an asset, if the asset you are trying to update does not exist the operation will throw a NotFoundException."}
{"id":10370,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"All assets must be wrapped in an entities array. You must provide the exact type name for the asset (case-sensitive)."}
{"id":10371,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"For a term, this is AtlasGlossaryTerm. You must provide the exact name of the existing asset (case-sensitive)."}
{"id":10372,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(Unless you want to change its name, in which case you can provide the new name instead.) You must provide the exact qualifiedName of the existing asset (case-sensitive). Must exactly match the qualifiedName of an existing asset If this does not exactly match the qualifiedName of an existing asset, the API call will instead create a new asset rather than updating an existing one."}
{"id":10373,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You must provide the exact qualifiedName of the existing asset (case-sensitive). Must exactly match the qualifiedName of an existing asset If this does not exactly match the qualifiedName of an existing asset, the API call will instead create a new asset rather than updating an existing one."}
{"id":10374,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"In this example, we're adding a certificate to the object. Note that you can include any number of enrichments together."}
{"id":10375,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Here we are also adding an announcement to the asset. Case-sensitive, exact match If you use a different capitalization or spelling for the qualifiedName , you may accidentally create a new asset rather than updating the existing one."}
{"id":10376,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"2 Remove information from an asset As mentioned in Enrich before updating section, only the information in your request will be updated on the object. But what if you want to remove some information that already exists on the asset in Atlan?"}
{"id":10377,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"We'll create an object we can take actions on from this updater. In this example, we'll remove any existing certificate from the object in Atlan."}
{"id":10378,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Note that you can chain any number of enrichments together. Here we are also removing any announcement from the asset."}
{"id":10379,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To persist the enrichment back to the object, we must build() the builder. Assign the result back Remember to assign the result of the build() operation back to your original object."}
{"id":10380,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Otherwise the result is not persisted back into any variable! (In this case we're assigning to the term variable back on line 5.) To persist the enrichment back to the object, we must build() the builder."}
{"id":10381,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Assign the result back Remember to assign the result of the build() operation back to your original object. Otherwise the result is not persisted back into any variable!"}
{"id":10382,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(In this case we're assigning to the term variable back on line 5.) The save() method will either: Update an existing asset, if Atlan already has a term with the same name and qualifiedName in the same glossary. Create a new asset, if Atlan does not have a term with the same name in the same glossary."}
{"id":10383,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant. The save() method will either: Update an existing asset, if Atlan already has a term with the same name and qualifiedName in the same glossary."}
{"id":10384,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Create a new asset, if Atlan does not have a term with the same name in the same glossary. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":10385,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: getCreatedAssets() lists assets that were created getUpdatedAssets() lists assets that were updated Note that the save() method always returns objects of type Asset , though. You can distinguish what was created or updated: getCreatedAssets() lists assets that were created getUpdatedAssets() lists assets that were updated Note that the save() method always returns objects of type Asset , though."}
{"id":10386,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10387,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10388,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"In this example we will remove an existing certificate from any existing certificate from the object. In this example we will remove any existing announcement from the object."}
{"id":10389,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"We'll create an object we can take actions on from this updater. In this example, we'll remove any existing certificate from the object in Atlan."}
{"id":10390,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Note that you can chain any number of enrichments together. Here we are also removing any announcement from the asset."}
{"id":10391,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To persist the enrichment back to the object, we must build() the builder. Assign the result back Remember to assign the result of the build() operation back to your original object."}
{"id":10392,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Otherwise the result is not persisted back into any variable! (In this case we're assigning to the term variable back on line 5.) To persist the enrichment back to the object, we must build() the builder."}
{"id":10393,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Assign the result back Remember to assign the result of the build() operation back to your original object. Otherwise the result is not persisted back into any variable!"}
{"id":10394,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(In this case we're assigning to the term variable back on line 5.) The save() method will either: Update an existing asset, if Atlan already has a term with the same name and qualifiedName in the same glossary. Create a new asset, if Atlan does not have a term with the same name in the same glossary."}
{"id":10395,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant. The save() method will either: Update an existing asset, if Atlan already has a term with the same name and qualifiedName in the same glossary."}
{"id":10396,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Create a new asset, if Atlan does not have a term with the same name in the same glossary. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant."}
{"id":10397,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You can distinguish what was created or updated: createdAssets lists assets that were created updatedAssets lists assets that were updated Note that the save() method always returns objects of type Asset , though. You can distinguish what was created or updated: createdAssets lists assets that were created updatedAssets lists assets that were updated Note that the save() method always returns objects of type Asset , though."}
{"id":10398,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10399,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"The Asset class is a superclass of all assets. So we need to cast to more specific types (like GlossaryTerm ) after verifying the object that was actually returned."}
{"id":10400,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"All assets must be wrapped in an entities array. You must provide the exact type name for the asset (case-sensitive)."}
{"id":10401,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"For a term, this is AtlasGlossaryTerm. You must provide the exact name of the existing asset (case-sensitive)."}
{"id":10402,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"(Unless you want to change its name, in which case you can provide the new name instead.) You must provide the exact qualifiedName of the existing asset (case-sensitive). Must exactly match the qualifiedName of an existing asset If this does not exactly match the qualifiedName of an existing asset, the API call will instead create a new asset rather than updating an existing one."}
{"id":10403,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"You must provide the exact qualifiedName of the existing asset (case-sensitive). Must exactly match the qualifiedName of an existing asset If this does not exactly match the qualifiedName of an existing asset, the API call will instead create a new asset rather than updating an existing one."}
{"id":10404,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"In this example, we're removing any existing certificate information from the object in Atlan (by sending null ). Note that you can include any number of enrichments together."}
{"id":10405,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Here we are also removing any announcement from the asset. Atlan automatically detects changes to determine whether to create or update an asset — see the Importance of identifiers for a more detailed explanation."}
{"id":10406,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To strictly update (and avoid creating) an asset, you must first look for the existing asset and only if found proceed with your update. When the SDKs provide such strict update functionality, this is what they are doing behind-the-scenes."}
{"id":10407,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Be aware that this will impact performance, so you should only do this where strictly necessary for your logic. ↩ Atlan automatically detects changes to determine whether to create or update an asset — see the Importance of identifiers for a more detailed explanation."}
{"id":10408,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"To strictly update (and avoid creating) an asset, you must first look for the existing asset and only if found proceed with your update. When the SDKs provide such strict update functionality, this is what they are doing behind-the-scenes."}
{"id":10409,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"Be aware that this will impact performance, so you should only do this where strictly necessary for your logic. ↩ This is because Atlan uses the exact qualifiedName to determine whether it should do an update."}
{"id":10410,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"For a more detailed explanation, see the Importance of identifiers. ↩ This is because Atlan uses the exact qualifiedName to determine whether it should do an update."}
{"id":10411,"url":"https:\/\/developer.atlan.com\/snippets\/advanced-examples\/update\/","title":"Updating an asset - Developer","text":"For a more detailed explanation, see the Importance of identifiers. ↩"}
{"id":10412,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10413,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10414,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10415,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10416,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10417,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10418,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10419,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10420,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10421,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10422,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10423,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10424,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10425,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10426,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10427,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10428,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10429,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10430,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10431,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10432,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10433,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10434,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10435,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10436,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10437,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10438,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10439,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10440,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10441,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10442,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10443,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10444,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10445,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10446,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10447,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10448,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10449,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10450,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10451,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10452,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10453,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10454,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10455,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10456,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10457,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10458,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10459,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10460,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10461,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10462,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10463,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10464,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10465,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10466,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10467,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10468,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10469,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10470,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10471,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10472,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10473,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10474,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10475,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10476,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10477,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10478,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10479,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10480,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10481,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10482,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10483,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10484,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10485,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10486,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10487,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10488,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10489,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10490,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10491,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10492,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10493,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10494,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10495,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10496,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10497,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10498,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10499,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10500,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10501,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10502,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10503,"url":"https:\/\/developer.atlan.com\/getting-started\/#importance-of-identifiers","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10504,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10505,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10506,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10507,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10508,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10509,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10510,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10511,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10512,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10513,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10514,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10515,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10516,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10517,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10518,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10519,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10520,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10521,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10522,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10523,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10524,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10525,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10526,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10527,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10528,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10529,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10530,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10531,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10532,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10533,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10534,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10535,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10536,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10537,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10538,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10539,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10540,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10541,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10542,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10543,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10544,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10545,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10546,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10547,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10548,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10549,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10550,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10551,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10552,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10553,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10554,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10555,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10556,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10557,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10558,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10559,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10560,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10561,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10562,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10563,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10564,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10565,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10566,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10567,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10568,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10569,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10570,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10571,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10572,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10573,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10574,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10575,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10576,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10577,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10578,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10579,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10580,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10581,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10582,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10583,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10584,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10585,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10586,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10587,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10588,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10589,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10590,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10591,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10592,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10593,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10594,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10595,"url":"https:\/\/developer.atlan.com\/getting-started\/#guid_1","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10596,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10597,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10598,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10599,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10600,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10601,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10602,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10603,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10604,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10605,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10606,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10607,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10608,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10609,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10610,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10611,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10612,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10613,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10614,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10615,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10616,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10617,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10618,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10619,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10620,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10621,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10622,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10623,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10624,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10625,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10626,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10627,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10628,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10629,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10630,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10631,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10632,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10633,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10634,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10635,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10636,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10637,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10638,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10639,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10640,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10641,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10642,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10643,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10644,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10645,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10646,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10647,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10648,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10649,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10650,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10651,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10652,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10653,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10654,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10655,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10656,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10657,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10658,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10659,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10660,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10661,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10662,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10663,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10664,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10665,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10666,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10667,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10668,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10669,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10670,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10671,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10672,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10673,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10674,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10675,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10676,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10677,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10678,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10679,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10680,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10681,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10682,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10683,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10684,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10685,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10686,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10687,"url":"https:\/\/developer.atlan.com\/getting-started\/#fn%3A2","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10688,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10689,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10690,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10691,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10692,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10693,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10694,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10695,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10696,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10697,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10698,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10699,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10700,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10701,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10702,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10703,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10704,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10705,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10706,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10707,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10708,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10709,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10710,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10711,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10712,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10713,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10714,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10715,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10716,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10717,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10718,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10719,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10720,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10721,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10722,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10723,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10724,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10725,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10726,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10727,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10728,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10729,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10730,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10731,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10732,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10733,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10734,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10735,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10736,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10737,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10738,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10739,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10740,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10741,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10742,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10743,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10744,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10745,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10746,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10747,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10748,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10749,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10750,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10751,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10752,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10753,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10754,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10755,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10756,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10757,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10758,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10759,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10760,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10761,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10762,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10763,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10764,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10765,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10766,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10767,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10768,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10769,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10770,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10771,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10772,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10773,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10774,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10775,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10776,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10777,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10778,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10779,"url":"https:\/\/developer.atlan.com\/getting-started\/#qualifiedname_1","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10780,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10781,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10782,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10783,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10784,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10785,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10786,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10787,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10788,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10789,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10790,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10791,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10792,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10793,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10794,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10795,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10796,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10797,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10798,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10799,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10800,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10801,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10802,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10803,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10804,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10805,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10806,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10807,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10808,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10809,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10810,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10811,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10812,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10813,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10814,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10815,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10816,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10817,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10818,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10819,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10820,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10821,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10822,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10823,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10824,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10825,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10826,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10827,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10828,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10829,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10830,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10831,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10832,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10833,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10834,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10835,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10836,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10837,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10838,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10839,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10840,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10841,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10842,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10843,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10844,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10845,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10846,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10847,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10848,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10849,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10850,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10851,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10852,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10853,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10854,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10855,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10856,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10857,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10858,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10859,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10860,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10861,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10862,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10863,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10864,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10865,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10866,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10867,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10868,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10869,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10870,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10871,"url":"https:\/\/developer.atlan.com\/getting-started\/#limit-to-changes-only","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10872,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":10873,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":10874,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10875,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":10876,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10877,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10878,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10879,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10880,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":10881,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10882,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":10883,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":10884,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":10885,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":10886,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":10887,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10888,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":10889,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":10890,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":10891,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":10892,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":10893,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":10894,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":10895,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":10896,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":10897,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":10898,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":10899,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":10900,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":10901,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":10902,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10903,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10904,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10905,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":10906,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":10907,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":10908,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":10909,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":10910,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":10911,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":10912,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":10913,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":10914,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":10915,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":10916,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":10917,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":10918,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":10919,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":10920,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":10921,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":10922,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":10923,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10924,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10925,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10926,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10927,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":10928,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":10929,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10930,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10931,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":10932,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":10933,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":10934,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":10935,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":10936,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":10937,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":10938,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":10939,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":10940,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":10941,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10942,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10943,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10944,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10945,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10946,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10947,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10948,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10949,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10950,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":10951,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":10952,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":10953,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":10954,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":10955,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":10956,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":10957,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":10958,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":10959,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":10960,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":10961,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":10962,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":10963,"url":"https:\/\/developer.atlan.com\/getting-started\/#bulk-changes","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":10964,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"End-to-end bulk update To walk through this using an example, and to compare and contrast the approaches, imagine you want to: Mark all views (including materialized views) in a particular schema as verified, unless they already have some certificate. Change the owner of the same views."}
{"id":10965,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The usual end-to-end pattern for updating many assets efficiently involves three steps: Finding the assets you want to update. Applying your updates to each asset (in-memory)."}
{"id":10966,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Sending those changes to Atlan (in batches). You can do each of these steps in sequence, for example: 1."}
{"id":10967,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Find the assets The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":10968,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"To start building up a query with multiple conditions, you can use the select() helper on any client's assets member. To start building up a query with multiple conditions, you can use the select() helper on any client's assets member."}
{"id":10969,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() method to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() method to restrict results to only views and materialized views."}
{"id":10970,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":10971,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":10972,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Here you can play around with different page sizes, to further limit API calls by retrieving more results per page. Here you can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":10973,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualified_name of every view starts with the qualified_name of its parent (schema), so we can limit the results to a particular schema by using the qualified_name. The qualified_name of every view starts with the qualified_name of its parent (schema), so we can limit the results to a particular schema by using the qualified_name."}
{"id":10974,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the CompoundQuery.asset_types() helper method to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the CompoundQuery.asset_types() helper method to restrict results to only views and materialized views."}
{"id":10975,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the where_not() method. This will exclude any assets where a certificate already has_any_value()."}
{"id":10976,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the where_not() method. This will exclude any assets where a certificate already has_any_value()."}
{"id":10977,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Here you can play around with different page sizes, to further limit API calls by retrieving more results per page. Here you can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":10978,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":10979,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"To start building up a query with multiple conditions, you can use the select() helper on any client's assets member. To start building up a query with multiple conditions, you can use the select() helper on any client's assets member."}
{"id":10980,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() helper method to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() helper method to restrict results to only views and materialized views."}
{"id":10981,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":10982,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":10983,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Here you can play around with different page sizes, to further limit API calls by retrieving more results per page. Here you can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":10984,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":10985,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use an exact match on multiple types to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use an exact match on multiple types to restrict results to only views and materialized views."}
{"id":10986,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the must_not clause. This will exclude any assets that already have a certificate present."}
{"id":10987,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the must_not clause. This will exclude any assets that already have a certificate present."}
{"id":10988,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When paging through results, you should specify a sort to give a stable set of results across pages. The most reliable sort will be by GUID of the asset, as this is guaranteed to be unique for every asset."}
{"id":10989,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When paging through results, you should specify a sort to give a stable set of results across pages. The most reliable sort will be by GUID of the asset, as this is guaranteed to be unique for every asset."}
{"id":10990,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Here you can play around with different page sizes, to further limit API calls by retrieving more results per page. Here you can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":10991,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"2. Build-up your changes Next, you iterate through those results and make the changes you want to each one."}
{"id":10992,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Use the multiple operations pattern to make multiple changes to each asset. Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":10993,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan Additional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":10994,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset."}
{"id":10995,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":10996,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":10997,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder."}
{"id":10998,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder. You can then add your (in-memory) modified asset to the batch."}
{"id":10999,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch. As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call."}
{"id":11000,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size."}
{"id":11001,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch."}
{"id":11002,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call. Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11003,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size. Create a batch of assets to accumulate changes across multiple assets before applying those changes in Atlan itself."}
{"id":11004,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch() takes the following parameters: client : an instance of AssetClient. max_size : the maximum size of each batch to be processed (per API call)."}
{"id":11005,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Additional optional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: replace_atlan_tags ( default: False ): If True replace all classifications (tags) on the assets in the batch, which would include removing classifications (tags) if none are provided for the assets in the batch itself (or False if you still want to ignore classifications) custom_metadata_handling ( default: CustomMetadataHandling.IGNORE ): control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) capture_failures ( default: False ): control whether failures should be captured across batches ( True ) or ignored ( False ) update_only ( default: False ): control whether the batch should only attempt to update assets that already exist ( True ) or also create assets if they do not yet exist ( False ) track ( default: False ): control whether details about each created and updated asset across batches should be tracked ( True ) or ignored ( False ) — counts will always be kept case_insensitive ( default: False ): control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( True ) or case-sensitively ( False ) creation_handling ( default: AssetCreationHandling.FULL ): control what kind of assets to create, if not running in update_only mode; PARTIAL assets (only available in lineage), or FULL assets table_view_agnostic ( default: False ): control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( False ), or if tables, views and materialized views should be treated interchangeably ( True ) Create a batch of assets to accumulate changes across multiple assets before applying those changes in Atlan itself."}
{"id":11006,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch() takes the following parameters: client : an instance of AssetClient. max_size : the maximum size of each batch to be processed (per API call)."}
{"id":11007,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: replace_atlan_tags ( default: False ): If True replace all classifications (tags) on the assets in the batch, which would include removing classifications (tags) if none are provided for the assets in the batch itself (or False if you still want to ignore classifications) custom_metadata_handling ( default: CustomMetadataHandling.IGNORE ): control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) capture_failures ( default: False ): control whether failures should be captured across batches ( True ) or ignored ( False ) update_only ( default: False ): control whether the batch should only attempt to update assets that already exist ( True ) or also create assets if they do not yet exist ( False ) track ( default: False ): control whether details about each created and updated asset across batches should be tracked ( True ) or ignored ( False ) — counts will always be kept case_insensitive ( default: False ): control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( True ) or case-sensitively ( False ) creation_handling ( default: AssetCreationHandling.FULL ): control what kind of assets to create, if not running in update_only mode; PARTIAL assets (only available in lineage), or FULL assets table_view_agnostic ( default: False ): control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( False ), or if tables, views and materialized views should be treated interchangeably ( True ) Every asset implements the trim_to_required() method, which gives you an object containing only the bare minimum information needed to update that asset."}
{"id":11008,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":11009,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trim_to_required() method, which gives you an object containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":11010,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply add those updates onto the trimmed object."}
{"id":11011,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply add those updates onto the trimmed object. You can then add your (in-memory) modified asset to the batch."}
{"id":11012,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch. As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call."}
{"id":11013,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size."}
{"id":11014,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch."}
{"id":11015,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call. Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11016,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size. Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11017,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan Additional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11018,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset."}
{"id":11019,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":11020,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":11021,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder."}
{"id":11022,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder. You can then add your (in-memory) modified asset to the batch."}
{"id":11023,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch. As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call."}
{"id":11024,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size."}
{"id":11025,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch."}
{"id":11026,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call. Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11027,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size. Up to your own code There are no API calls to make to change the results in-memory."}
{"id":11028,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"How you implement this will be entirely up to how you are writing your code. 3."}
{"id":11029,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Save them in batches Finally, send the changes you have queued up in batches. Use the multiple assets pattern to update multiple assets at the same time."}
{"id":11030,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11031,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11032,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11033,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan. Either can therefore also run into trouble and raise an error through an AtlanException."}
{"id":11034,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"It is up to you to handle such potential errors as you see fit. Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan."}
{"id":11035,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Either can therefore also run into trouble and raise an error through an AtlanException. It is up to you to handle such potential errors as you see fit."}
{"id":11036,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11037,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. The Batch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11038,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11039,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Both the .add() and .flush() operations of the Batch could send a request over to Atlan. Either can therefore also run into trouble and raise an error through an AtlanError."}
{"id":11040,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"It is up to you to handle such potential errors as you see fit. Both the .add() and .flush() operations of the Batch could send a request over to Atlan."}
{"id":11041,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Either can therefore also run into trouble and raise an error through an AtlanError. It is up to you to handle such potential errors as you see fit."}
{"id":11042,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11043,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11044,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11045,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan. Either can therefore also run into trouble and raise an error through an AtlanException."}
{"id":11046,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"It is up to you to handle such potential errors as you see fit. Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan."}
{"id":11047,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Either can therefore also run into trouble and raise an error through an AtlanException. It is up to you to handle such potential errors as you see fit."}
{"id":11048,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"All details must still be included in an outer entities array. You need to specify the type for each asset you are updating."}
{"id":11049,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You need to specify other required attributes for each asset, such as its name and qualifiedName. Add on any other attributes or relationships you want to set on the asset, such as in the running example a verified certificate and new individual owner."}
{"id":11050,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Add another object to the payload to represent another asset that should be updated by the same API call. Once again specify all the required information for that kind of asset, and any of the details for attributes or relationships you want to set."}
{"id":11051,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Alternatively, when using an SDK, you can pipeline these operations together. The pipeline will run just as efficiently as the step-by-step approach above: Lazily-fetching each page of results Batching up and bulk-saving changes The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":11052,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11053,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When parallel-processing (see further notes on the stream(true) ) you need to use a parallel-capable ParallelBatch : The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan Additional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11054,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When parallel-processing (see further notes on the stream(true) ) you need to use a parallel-capable ParallelBatch : The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) You can then start defining a pipeline directly against the client's assets by using the select() method."}
{"id":11055,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Including archived (soft-deleted) assets Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() method to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in() method to restrict results to only views and materialized views."}
{"id":11056,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":11057,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":11058,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"(Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page. (Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":11059,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Once you have defined the criteria for your pipeline, call the stream() method to push-down the pipeline to Atlan. This will: Page through the results by lazily fetching each subsequent page as you iterate through them."}
{"id":11060,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"(So if you use a limit() on the stream, for example, you can break out before retrieving all pages.) Can also run in parallel threads You can also parallel-stream the results by passing true to the stream() method. This will spawn multiple threads that each independently process a page of results and combine the results in parallel."}
{"id":11061,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"While this can be significantly faster for processing many results, keep in mind if you are collecting the results into any structure that structure must be thread-safe. (For example, you'll need to use things like ConcurrentHashMap rather than just HashMap , and to use ParallelBatch rather than AssetBatch if making changes.) For each result, you can then carry out your changes and submit them into the batch."}
{"id":11062,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"For each result, you can then carry out your changes and submit them into the batch. Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset."}
{"id":11063,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":11064,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":11065,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder."}
{"id":11066,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder. You can then add your (in-memory) modified asset to the batch."}
{"id":11067,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch. As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call."}
{"id":11068,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size."}
{"id":11069,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch."}
{"id":11070,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call. Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11071,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size. Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan."}
{"id":11072,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Either can therefore also run into trouble and raise an error through an AtlanException. It is up to you to handle such potential errors as you see fit."}
{"id":11073,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan. Either can therefore also run into trouble and raise an error through an AtlanException."}
{"id":11074,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"It is up to you to handle such potential errors as you see fit. The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11075,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11076,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11077,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":11078,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. Create a batch of assets to accumulate changes across multiple assets before applying those changes in Atlan itself."}
{"id":11079,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch() takes the following parameters: client : an instance of AssetClient. max_size : the maximum size of each batch to be processed (per API call)."}
{"id":11080,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Additional optional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: replace_atlan_tags ( default: False ): If True replace all classifications (tags) on the assets in the batch, which would include removing classifications (tags) if none are provided for the assets in the batch itself (or False if you still want to ignore classifications) custom_metadata_handling ( default: CustomMetadataHandling.IGNORE ): control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) capture_failures ( default: False ): control whether failures should be captured across batches ( True ) or ignored ( False ) update_only ( default: False ): control whether the batch should only attempt to update assets that already exist ( True ) or also create assets if they do not yet exist ( False ) track ( default: False ): control whether details about each created and updated asset across batches should be tracked ( True ) or ignored ( False ) — counts will always be kept case_insensitive ( default: False ): control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( True ) or case-sensitively ( False ) creation_handling ( default: AssetCreationHandling.FULL ): control what kind of assets to create, if not running in update_only mode; PARTIAL assets (only available in lineage), or FULL assets table_view_agnostic ( default: False ): control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( False ), or if tables, views and materialized views should be treated interchangeably ( True ) Create a batch of assets to accumulate changes across multiple assets before applying those changes in Atlan itself."}
{"id":11081,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch() takes the following parameters: client : an instance of AssetClient. max_size : the maximum size of each batch to be processed (per API call)."}
{"id":11082,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: replace_atlan_tags ( default: False ): If True replace all classifications (tags) on the assets in the batch, which would include removing classifications (tags) if none are provided for the assets in the batch itself (or False if you still want to ignore classifications) custom_metadata_handling ( default: CustomMetadataHandling.IGNORE ): control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) capture_failures ( default: False ): control whether failures should be captured across batches ( True ) or ignored ( False ) update_only ( default: False ): control whether the batch should only attempt to update assets that already exist ( True ) or also create assets if they do not yet exist ( False ) track ( default: False ): control whether details about each created and updated asset across batches should be tracked ( True ) or ignored ( False ) — counts will always be kept case_insensitive ( default: False ): control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( True ) or case-sensitively ( False ) creation_handling ( default: AssetCreationHandling.FULL ): control what kind of assets to create, if not running in update_only mode; PARTIAL assets (only available in lineage), or FULL assets table_view_agnostic ( default: False ): control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( False ), or if tables, views and materialized views should be treated interchangeably ( True ) Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the CompoundQuery.asset_types() helper method to restrict results to only views and materialized views."}
{"id":11083,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the CompoundQuery.asset_types() helper method to restrict results to only views and materialized views. Since you only want to update views that do not already have a certificate, you can further limit the results using the where_not() method."}
{"id":11084,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will exclude any assets where a certificate already has_any_value(). Since you only want to update views that do not already have a certificate, you can further limit the results using the where_not() method."}
{"id":11085,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will exclude any assets where a certificate already has_any_value(). (Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":11086,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"(Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page. For each result, you can then carry out your changes and submit them into the batch."}
{"id":11087,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"For each result, you can then carry out your changes and submit them into the batch. Every asset implements the trim_to_required() method, which gives you a builder containing only the bare minimum information needed to update that asset."}
{"id":11088,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trim_to_required() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":11089,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trim_to_required() method, which gives you a builder containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":11090,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trim_to_required() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply set those updates on the trimmed object."}
{"id":11091,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply set those updates on the trimmed object. You can then add your (in-memory) modified asset to the batch."}
{"id":11092,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. The Batch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11093,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11094,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The Batch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11095,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName."}
{"id":11096,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The qualifiedName of every view starts with the qualifiedName of its parent (schema), so we can limit the results to a particular schema by using the qualifiedName. Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11097,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When parallel-processing (see further notes on the stream(true) ) you need to use a parallel-capable ParallelBatch : The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan Additional parameters By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) Create a batch of assets to build-up the changes across multiple assets before applying those changes in Atlan itself."}
{"id":11098,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"When parallel-processing (see further notes on the stream(true) ) you need to use a parallel-capable ParallelBatch : The first parameter defines the Atlan tenant on which the batch will be processed The second specifies the maximum number of assets to build-up before sending them across to Atlan By default (using only the options above) no classifications or custom metadata will be added or changed on the assets in each batch. To also include classifications and custom metadata, you need to use these additional parameters: A third parameter of true to replace all classifications on the assets in the batch, which would include removing classifications if none are provided for the assets in the batch itself (or false if you still want to ignore classifications) A fourth parameter to control how custom metadata should be handled for the assets: IGNORE any custom metadata changes in the batch, OVERWRITE to replace all custom metadata with what's provided in the batch (including removing custom metadata that already exists on an asset), or MERGE to only add or update custom metadata based on what's in the batch (leaving other existing custom metadata unchanged) a fifth parameter to control whether failures should be captured across batches ( true ) or ignored ( false ) a sixth parameter to control whether the batch should only attempt to update assets that already exist ( true ) or also create assets if they do not yet exist ( false ) a seventh parameter to control whether details about each created and updated asset across batches should be tracked ( true ) or ignored ( false ) — counts will always be kept an eighth parameter to control whether the matching for determining whether an asset already exists should be done in a case-insensitive way ( true ) or case-sensitively ( false ) a ninth parameter to control what kind of assets to create, if not running in updateOnly mode: partial assets (only available in lineage), or full assets a tenth parameter to control whether the matching for determining whether an asset already exists should be done strictly according to the data type specified ( false ), or if tables, views and materialized views should be treated interchangeably ( true ) You can then start defining a pipeline directly against the client's assets by using the select() method."}
{"id":11099,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Including archived (soft-deleted) assets Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in helper method to restrict results to only views and materialized views. Since there could be tables, views, materialized views and columns in this schema — but you only want views and materialized views — you can use the Asset.TYPE_NAME.in helper method to restrict results to only views and materialized views."}
{"id":11100,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":11101,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Since you only want to update views that do not already have a certificate, you can further limit the results using the whereNot() method. This will exclude any assets where a certificate already hasAnyValue()."}
{"id":11102,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"(Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page. (Optional) You can play around with different page sizes, to further limit API calls by retrieving more results per page."}
{"id":11103,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Once you have defined the criteria for your pipeline, call the stream() method to push-down the pipeline to Atlan. This will: Page through the results by lazily fetching each subsequent page as you iterate through them."}
{"id":11104,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"(So if you use a limit() on the stream, for example, you can break out before retrieving all pages.) Can also run in parallel threads You can also parallel-stream the results by passing true to the stream() method. This will spawn multiple threads that each independently process a page of results and combine the results in parallel."}
{"id":11105,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"While this can be significantly faster for processing many results, keep in mind if you are collecting the results into any structure that structure must be thread-safe. (For example, you'll need to use things like ConcurrentHashMap rather than just HashMap , and to use ParallelBatch rather than AssetBatch if making changes.) For each result, you can then carry out your changes and submit them into the batch."}
{"id":11106,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"For each result, you can then carry out your changes and submit them into the batch. Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset."}
{"id":11107,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan. By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update."}
{"id":11108,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Every asset implements the trimToRequired() method, which gives you a builder containing only the bare minimum information needed to update that asset. Limit your asset to only what you intend to update When you send an update to Atlan, it will only attempt to change the information you send in your request — leaving any information not in your request as-is (unchanged) on the asset in Atlan."}
{"id":11109,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"By using trimToRequired() you can remove all information you do not want to update, and then chain on only the details you do want to update. In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder."}
{"id":11110,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"In this running example, you are updating the certificate to verified and setting a new owner — so you simply chain those updates onto the trimmed builder. You can then add your (in-memory) modified asset to the batch."}
{"id":11111,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch. As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call."}
{"id":11112,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size."}
{"id":11113,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"You can then add your (in-memory) modified asset to the batch. Auto-saves as it goes As long as the number of assets built-up is below the maximum batch size specified when creating the batch, this will simply continue to build up the batch."}
{"id":11114,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"As soon as you hit the size limit for the batch, though, this same method will call the save() operation to batch-update all of those assets in a single API call. Remember to flush Since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11115,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up, when the size of the queue did not yet reach the full batch size. Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan."}
{"id":11116,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Either can therefore also run into trouble and raise an error through an AtlanException. It is up to you to handle such potential errors as you see fit."}
{"id":11117,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Both the .add() and .flush() operations of the AssetBatch could send a request over to Atlan. Either can therefore also run into trouble and raise an error through an AtlanException."}
{"id":11118,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"It is up to you to handle such potential errors as you see fit. The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size."}
{"id":11119,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch. This will send any remaining assets that were queued up."}
{"id":11120,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"The AssetBatch 's add() method used in the previous step will automatically save as its internal queue of assets reaches a full batch size. Remember to flush However, since your loop could finish before you reach another full batch, you must always remember to flush() the batch."}
{"id":11121,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"This will send any remaining assets that were queued up. Requires numerous API calls To implement the same logic purely through raw API calls will require making many calls: To page through the results."}
{"id":11122,"url":"https:\/\/developer.atlan.com\/patterns\/bulk\/end-to-end\/","title":"End-to-end bulk update - Developer","text":"To batch up a set of assets to update. To submit each batch of assets to update."}
{"id":11123,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":11124,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":11125,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11126,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":11127,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11128,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11129,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11130,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11131,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":11132,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11133,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11134,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":11135,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11136,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11137,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11138,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11139,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":11140,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11141,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":11142,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":11143,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":11144,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":11145,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":11146,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":11147,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":11148,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":11149,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":11150,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":11151,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":11152,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":11153,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11154,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11155,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11156,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11157,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":11158,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":11159,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":11160,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":11161,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":11162,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":11163,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":11164,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":11165,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":11166,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":11167,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":11168,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":11169,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":11170,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":11171,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":11172,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":11173,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":11174,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11175,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11176,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11177,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11178,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":11179,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":11180,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11181,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11182,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11183,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11184,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11185,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11186,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":11187,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":11188,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":11189,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":11190,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":11191,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":11192,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11193,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11194,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11195,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11196,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11197,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11198,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11199,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11200,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11201,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11202,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11203,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11204,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11205,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":11206,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11207,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11208,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11209,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11210,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":11211,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":11212,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":11213,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":11214,"url":"https:\/\/developer.atlan.com\/getting-started\/#where-to-go-from-here","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":11215,"url":"https:\/\/developer.atlan.com\/snippets\/","title":"Common tasks - Developer","text":"As developers ourselves, we know that learning is often easiest through example. In this Common tasks section you will find examples and explanations of the most common tasks in Atlan — those that are available on all assets."}
{"id":11216,"url":"https:\/\/developer.atlan.com\/snippets\/","title":"Common tasks - Developer","text":"However, if you want to find out how to do something unique to a particular kind of asset, or to do something orthogonal to assets, you may also want to explore: Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11217,"url":"https:\/\/developer.atlan.com\/snippets\/","title":"Common tasks - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11218,"url":"https:\/\/developer.atlan.com\/snippets\/","title":"Common tasks - Developer","text":"Manage governance structures"}
{"id":11219,"url":"https:\/\/developer.atlan.com\/patterns\/","title":"Asset-specific - Developer","text":"As developers ourselves, we know that learning is often easiest through example. In this Asset-specific operations section you will find details about managing specific kinds of assets."}
{"id":11220,"url":"https:\/\/developer.atlan.com\/patterns\/","title":"Asset-specific - Developer","text":"You may also want to explore: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11221,"url":"https:\/\/developer.atlan.com\/patterns\/","title":"Asset-specific - Developer","text":"Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11222,"url":"https:\/\/developer.atlan.com\/patterns\/","title":"Asset-specific - Developer","text":"Manage governance structures"}
{"id":11223,"url":"https:\/\/developer.atlan.com\/governance\/","title":"Governance structures - Developer","text":"As developers ourselves, we know that learning is often easiest through example. In this Governance structures operations section you will find details about managing the structures that can be used to govern your assets, rather than managing the assets themselves."}
{"id":11224,"url":"https:\/\/developer.atlan.com\/governance\/","title":"Governance structures - Developer","text":"You may also want to explore: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11225,"url":"https:\/\/developer.atlan.com\/governance\/","title":"Governance structures - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11226,"url":"https:\/\/developer.atlan.com\/governance\/","title":"Governance structures - Developer","text":"Focus on a specific kind of asset"}
{"id":11227,"url":"https:\/\/developer.atlan.com\/search\/","title":"Introduction to searching in Atlan - Developer","text":"Then, you can get a bit more advanced by adding in: Whether you want to sort those results How to page the results, when there are many of them Whether you want to aggregate any information from those results"}
{"id":11228,"url":"https:\/\/developer.atlan.com\/events\/","title":"Events overview - Developer","text":"Atlan produces events when any of the following activities occur in the system: An asset is created An asset is updated An asset is deleted Custom metadata is added to an asset Custom metadata is removed from an asset An asset is tagged An asset is untagged Lineage is created Each scenario above can be identified by Atlan producing one or more event payloads:"}
{"id":11229,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":11230,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":11231,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11232,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":11233,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11234,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11235,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11236,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11237,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":11238,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11239,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11240,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":11241,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11242,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11243,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11244,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11245,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":11246,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11247,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":11248,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":11249,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":11250,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":11251,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":11252,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":11253,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":11254,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":11255,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":11256,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":11257,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":11258,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":11259,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11260,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11261,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11262,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11263,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":11264,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":11265,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":11266,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":11267,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":11268,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":11269,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":11270,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":11271,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":11272,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":11273,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":11274,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":11275,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":11276,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":11277,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":11278,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":11279,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":11280,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11281,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11282,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11283,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11284,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":11285,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":11286,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11287,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11288,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11289,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11290,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11291,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11292,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":11293,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":11294,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":11295,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":11296,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":11297,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":11298,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11299,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11300,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11301,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11302,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11303,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11304,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11305,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11306,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11307,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11308,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11309,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11310,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11311,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":11312,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11313,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11314,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11315,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11316,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":11317,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":11318,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":11319,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":11320,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A1","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":11321,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"An introductory walkthrough You might also like our Atlan Platform Essentials certification. Not sure where to start?"}
{"id":11322,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process."}
{"id":11323,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11324,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":11325,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11326,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11327,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11328,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11329,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on PyPI. You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter."}
{"id":11330,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11331,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like."}
{"id":11332,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil."}
{"id":11333,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters."}
{"id":11334,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter."}
{"id":11335,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e."}
{"id":11336,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11337,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters."}
{"id":11338,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging."}
{"id":11339,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code!"}
{"id":11340,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets."}
{"id":11341,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day."}
{"id":11342,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on."}
{"id":11343,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers."}
{"id":11344,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID."}
{"id":11345,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name. (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName."}
{"id":11346,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName."}
{"id":11347,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already."}
{"id":11348,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset."}
{"id":11349,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these."}
{"id":11350,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute()."}
{"id":11351,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11352,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11353,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11354,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets."}
{"id":11355,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata?"}
{"id":11356,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen?"}
{"id":11357,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan."}
{"id":11358,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally."}
{"id":11359,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID."}
{"id":11360,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems)."}
{"id":11361,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one."}
{"id":11362,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan."}
{"id":11363,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName."}
{"id":11364,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset."}
{"id":11365,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"a\/b\/c\/d is not the same as a\/B\/c\/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all?"}
{"id":11366,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset."}
{"id":11367,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Having a common \"identity\" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan."}
{"id":11368,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The crawler will not create duplicate tables every time it runs. This gives idempotency."}
{"id":11369,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage."}
{"id":11370,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan."}
{"id":11371,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it."}
{"id":11372,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11373,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11374,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11375,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11376,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name. Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset."}
{"id":11377,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED."}
{"id":11378,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11379,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11380,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want."}
{"id":11381,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. At the end of your chain of updates, you need to build the builder (into an object, in-memory)."}
{"id":11382,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes."}
{"id":11383,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName. Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset."}
{"id":11384,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want."}
{"id":11385,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"In this example, we change the certificate status to VERIFIED. And then, finally, you need to .Save() that object to persist those changes in Atlan."}
{"id":11386,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates."}
{"id":11387,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is."}
{"id":11388,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent."}
{"id":11389,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch."}
{"id":11390,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11391,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11392,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11393,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11394,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11395,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11396,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11397,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11398,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11399,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch."}
{"id":11400,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with)."}
{"id":11401,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example)."}
{"id":11402,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not \"full\"."}
{"id":11403,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets."}
{"id":11404,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11405,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11406,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11407,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11408,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible."}
{"id":11409,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site."}
{"id":11410,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive."}
{"id":11411,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet."}
{"id":11412,"url":"https:\/\/developer.atlan.com\/getting-started\/#fnref%3A2","title":"Introductory walkthrough - Developer","text":"(And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩"}
{"id":11413,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Throughout the portal you can focus on your preferred integration approach (and switch between them as you like): CLI Use the Atlan CLI to manage data contracts for assets in Atlan. Get started with CLI Use the Atlan CLI to manage data contracts for assets in Atlan."}
{"id":11414,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with CLI dbt Use dbt's meta field to enrich metadata resources straight from dbt into Atlan. Get started with dbt Use dbt's meta field to enrich metadata resources straight from dbt into Atlan."}
{"id":11415,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with dbt Java Pull our Java SDK from Maven Central, just like any other dependency. Get started with Java Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":11416,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Java Python Pull our Python SDK from PyPI, just like any other dependency. Get started with Python Pull our Python SDK from PyPI, just like any other dependency."}
{"id":11417,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Python Kotlin Pull our Java SDK from Maven Central, just like any other dependency. Get started with Kotlin Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":11418,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Kotlin Scala Pull our Java SDK from Maven Central, just like any other dependency. Get started with Scala Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":11419,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Scala Clojure Pull our Java SDK from Maven Central, just like any other dependency. Get started with Clojure Pull our Java SDK from Maven Central, just like any other dependency."}
{"id":11420,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Clojure Go Pull our Go SDK from GitHub, just like any other dependency. Get started with Go Pull our Go SDK from GitHub, just like any other dependency."}
{"id":11421,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with Go Events Tap into events Atlan produces to take immediate action, as metadata changes. Get started with events Tap into events Atlan produces to take immediate action, as metadata changes."}
{"id":11422,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with events Raw REST API You can call directly into our REST API, though we would recommend the SDKs. Get started with raw REST APIs Raw REST API You can call directly into our REST API, though we would recommend the SDKs."}
{"id":11423,"url":"https:\/\/developer.atlan.com\/sdks\/#integration-options","title":"Integration options - Developer","text":"Get started with raw REST APIs"}
{"id":11424,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"Limited functionality (so far) You can use Atlan's command-line interface (CLI) to manage some metadata in Atlan. Currently data contracts and metadata for a limited set of asset types can be managed through the CLI."}
{"id":11425,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"Obtain the CLI For now, the CLI must be downloaded as a pre-built binary: Disclaimer — closed preview When installed via Homebrew, you can easily keep things up-to-date. If you do not use it already, see Homebrew's own installation documents for setting up Homebrew itself."}
{"id":11426,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"Configure the CLI You can configure the CLI using a config file or in some cases environment variables, with the following minimum settings 1 : An API token that has access to your assets. The base URL of your tenant (including the https:\/\/ )."}
{"id":11427,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"(Optional) Enable logging to produce more details on what the CLI is doing. When logging is enabled, specify the level of verbosity."}
{"id":11428,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"An API token that has access to your assets. Define data sources You should also define data sources in the config file: Each data source definition must start with data_source , followed by a space and a unique reference name for the data source ( snowflake in this example)."}
{"id":11429,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"Reference name is your choice The reference name you give in the configuration file is only used here and as a reference in any data contracts you define. It need not match the name of the connection or data source in Atlan itself."}
{"id":11430,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"Each data source definition must start with data_source , followed by a space and a unique reference name for the data source ( snowflake in this example). Reference name is your choice The reference name you give in the configuration file is only used here and as a reference in any data contracts you define."}
{"id":11431,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"It need not match the name of the connection or data source in Atlan itself. You must indicate the type of connector for the data source (see connector types for options)."}
{"id":11432,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"You must indicate the type of connector for the data source (see connector types for options). Details of the connection must also be provided."}
{"id":11433,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"You must provide the name of the connection, as it appears in Atlan. You must provide the unique qualified_name of the connection in Atlan."}
{"id":11434,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"(Optional) You can also specify the database to use for this connection's assets by default, if none is specified in the data contract. (Optional) You can also specify the schema to use for this connection's assets by default, if none is specified in the data contract."}
{"id":11435,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"These ensure the CLI can map the details you specify in your data contract to the appropriate corresponding asset in Atlan. With the CLI, you can: Manage data contracts Upload and download files from Atlan's backing object store Sync metadata to a limited set of asset types Integrate data contracts with CI\/CD processing When both are specified, environment variables will take precedence."}
{"id":11436,"url":"https:\/\/developer.atlan.com\/sdks\/cli\/","title":"Atlan CLI - Developer","text":"↩ When both are specified, environment variables will take precedence. ↩"}
{"id":11437,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"See it in action in our automated enrichment course (45 mins). You can use dbt's meta field to enrich metadata resources from dbt into Atlan."}
{"id":11438,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"Atlan will ingest the information from this field and update the assets in Atlan accordingly. With this, you have a powerful way to keep the dbt assets documented directly as part of your dbt work."}
{"id":11439,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"The following is an example: The description at the top level of an asset defined in dbt will already be mapped to the description field for that asset in Atlan. More detailed metadata, however, needs to be specified within the meta field."}
{"id":11440,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":". and within the meta field, further within the atlan sub-field."}
{"id":11441,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"For attributes, such as certificates, announcements, or owners these need to be specified within the attributes sub-field. Classifications need to be specified within a classifications sub-field."}
{"id":11442,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"Note that the meta field and its sub-structure (including all the detailed attributes) can also be applied to columns within a model. This rich metadata will then be loaded to the corresponding attributes on the asset in Atlan."}
{"id":11443,"url":"https:\/\/developer.atlan.com\/sdks\/dbt\/","title":"dbt - Developer","text":"For more details on specific examples, see the dbt tabs in the Common asset actions snippets."}
{"id":11444,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency."}
{"id":11445,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes."}
{"id":11446,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"You can include logback as a simple binding mechanism to send any logging information out to your console (standard out), at INFO -level or above. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":11447,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system."}
{"id":11448,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"But it's your choice exactly how you manage the API token and including it for use within the client. (Note that you can also explicity provide only the tenant URL, and the constructor will look for only the API key through an environment variable.) That's it — once these are set you can start using your SDK to make live calls against your Atlan instance!"}
{"id":11449,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11450,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11451,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11452,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11453,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11454,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException."}
{"id":11455,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":11456,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":11457,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems."}
{"id":11458,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems."}
{"id":11459,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the ch.qos.logback:logback-classic:1.2.11 logback binding with log4j2 bindings."}
{"id":11460,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException. When there is a 403 response indicating that permission for an operation is not (yet) available."}
{"id":11461,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":11462,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":11463,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"(This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out."}
{"id":11464,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client."}
{"id":11465,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds)."}
{"id":11466,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant."}
{"id":11467,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual."}
{"id":11468,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"You can access the operations for assets directly on the client, under client.assets. These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides."}
{"id":11469,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request."}
{"id":11470,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches."}
{"id":11471,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant."}
{"id":11472,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java. command."}
{"id":11473,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument."}
{"id":11474,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":". and your password as the second argument, as a char[]."}
{"id":11475,"url":"https:\/\/developer.atlan.com\/sdks\/java\/","title":"Java SDK - Developer","text":"(Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":11476,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is currently available on pypi."}
{"id":11477,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"You can use pip to install it as follows: Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token!"}
{"id":11478,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system. But it's your choice exactly how you manage the API token and including it for use within the client."}
{"id":11479,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"In some scenarios, you may not want to expose the entire API token or manage environment variables. Instead, you can provide the GUID of the API token, and the SDK will internally fetch the actual access token."}
{"id":11480,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"When to use this approach: Building apps that use the SDK where token security is a concern When you want to avoid exposing full API tokens in your configuration For containerized applications that need secure token management Before using this approach, ensure your Argo template is configured with CLIENT_ID and CLIENT_SECRET : Create client from token GUID : Use AtlanClient.from_token_guid() to create a client using the GUID of an API token. The SDK will automatically fetch the actual access token using the configured CLIENT_ID and CLIENT_SECRET."}
{"id":11481,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"That's it — once these are set you can start using your SDK to make live calls against your Atlan instance! 🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets."}
{"id":11482,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11483,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11484,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11485,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11486,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above."}
{"id":11487,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError. For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection."}
{"id":11488,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place."}
{"id":11489,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionError will be raised."}
{"id":11490,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":11491,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses logging module of the standard library that can provide a flexible framework for emitting log messages."}
{"id":11492,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). logging.INFO : used to confirm that things are working as expected."}
{"id":11493,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future. logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":11494,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now). You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK)."}
{"id":11495,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"logging.INFO : used to confirm that things are working as expected. logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future."}
{"id":11496,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation. logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now)."}
{"id":11497,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"By default, logs will appear in your console. If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file."}
{"id":11498,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"\/tmp\/pyatlan.json : log file in JSON format. By default, logs will appear in your console."}
{"id":11499,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":11500,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"logging.config.fileConfig('pyatlan\/logging.conf') : this will generate logs according to the configuration defined in pyatlan\/logging.conf and will generate two log files: \/tmp\/pyatlan.log : default log file. \/tmp\/pyatlan.json : log file in JSON format."}
{"id":11501,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 429 response indicating that the request rate limit has been exceeded, and you need to retry after some time."}
{"id":11502,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"When there is a 50x response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":11503,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Before each retry, the SDK will apply a delay using an exponential backoff. (Currently the values for the exponential backoff are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":11504,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"(This is set to 5 by default.) By default, the SDK AtlanClient() has the following timeout settings: read_timeout : 900.0 seconds ( 15 minutes) connect_timeout : 30.0 seconds If you need to override these defaults, you can do so as shown in the example below: Since version 1.0.0, the Python SDK supports connecting to multiple tenants.[^1] When you use the AtlanClient() method you are actually setting a default client. This default client will be used behind-the-scenes for any operations that need information specific to an Atlan tenant."}
{"id":11505,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"When you want to override that default client you can create a new one and use the set_default_client() method to change it: The AtlanClient() method will return a client for the given base URL, creating a new client and setting this new client as the default client. If you want to switch between clients that you have already created, you can use Atlan.set_default_client() to change between them."}
{"id":11506,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume."}
{"id":11507,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. (And since in the majority of use cases you only need access to a single tenant, this means you can most likely just rely on the default client and the fallback behavior.) Pyatlan uses the Requests library which supports proxy configuration via environment variables."}
{"id":11508,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Requests relies on the proxy configuration defined by standard environment variables http_proxy, https_proxy, no_proxy, and all_proxy. Uppercase variants of these variables are also supported."}
{"id":11509,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"You can therefore set them to configure Pyatlan (only set the ones relevant to your needs): To use HTTP Basic Auth with your proxy, use the http:\/\/user:password@host\/ syntax in any of the above configuration entries: Currently, the way this is implemented limits you to either avoiding multiple threads in your Python code (if you need to use multiple clients), or if you want to use multiple threads you should only use a single client. Asynchronous SDK operations To get started, you need to initialize an AsyncAtlanClient : Create an async client using the same configuration pattern as the synchronous client."}
{"id":11510,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"Concurrent operations for improved performance The real power of async comes from running multiple operations concurrently. Instead of waiting for each operation to complete sequentially, you can execute them in parallel and reduce total execution time: Synchronous : Total time = operation₁ + operation₂ +."}
{"id":11511,"url":"https:\/\/developer.atlan.com\/sdks\/python\/","title":"Python SDK - Developer","text":"+ operationₙ Asynchronous : Total time = max(operation₁, operation₂,. , operationₙ) When to use async Async is most beneficial when you have: Multiple independent operations that can run concurrently I\/O-heavy workloads like API calls, database queries, or file operations Long-running operations where parallelization provides significant time savings For simple, single operations, the synchronous client may be more straightforward to use."}
{"id":11512,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Obtain the SDK For Kotlin, you can reuse the existing Java SDK as-is. It is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency."}
{"id":11513,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes."}
{"id":11514,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":11515,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11516,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11517,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11518,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11519,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11520,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException."}
{"id":11521,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":11522,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":11523,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems."}
{"id":11524,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems."}
{"id":11525,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the org.slf4j:slf4j-simple:2.0.7 binding with log4j2 bindings."}
{"id":11526,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException. When there is a 403 response indicating that permission for an operation is not (yet) available."}
{"id":11527,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried."}
{"id":11528,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted."}
{"id":11529,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"(This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out."}
{"id":11530,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client."}
{"id":11531,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds)."}
{"id":11532,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant."}
{"id":11533,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual."}
{"id":11534,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"You can access the operations for assets directly on the client, under client.assets. These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides."}
{"id":11535,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request."}
{"id":11536,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches."}
{"id":11537,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant."}
{"id":11538,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java. command."}
{"id":11539,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument."}
{"id":11540,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":". and your password as the second argument, as a char[]."}
{"id":11541,"url":"https:\/\/developer.atlan.com\/sdks\/kotlin\/","title":"Kotlin SDK - Developer","text":"(Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":11542,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Obtain the SDK For Scala, you can reuse the existing Java SDK as-is. It is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency (replace the version number with the latest version indicated in the badges above)."}
{"id":11543,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include the scala-logging utility as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":11544,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets."}
{"id":11545,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11546,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11547,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11548,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11549,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above."}
{"id":11550,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException. For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection."}
{"id":11551,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place."}
{"id":11552,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionException will be raised."}
{"id":11553,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":11554,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems. The SDK uses slf4j to be logging framework-agnostic."}
{"id":11555,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"You can therefore configure your own preferred logging framework: Add log4j2 bindings. The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException."}
{"id":11556,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 500 response indicating that something went wrong on the server side."}
{"id":11557,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"If any request encounters one of these problems, it will be retried. Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s."}
{"id":11558,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"(Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted. (This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client."}
{"id":11559,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out. By default, this is set to 80 seconds."}
{"id":11560,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client. Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds."}
{"id":11561,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds). Since version 0.9.0, the Java SDK supports connecting to multiple tenants."}
{"id":11562,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant. You can also (optionally) provide a second argument to directly give the API token for the tenant."}
{"id":11563,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Create an object as usual. You can access the operations for assets directly on the client, under client.assets."}
{"id":11564,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides. Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request."}
{"id":11565,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request. Alternatively, you can pass the client to the operation on the object itself."}
{"id":11566,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume."}
{"id":11567,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java."}
{"id":11568,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"command. These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object."}
{"id":11569,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"Provide your username as the first argument. ."}
{"id":11570,"url":"https:\/\/developer.atlan.com\/sdks\/scala\/","title":"Scala SDK - Developer","text":"and your password as the second argument, as a char[]. (Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":11571,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Obtain the SDK For Clojure, you can reuse the existing Java SDK as-is. It is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency (replace the version number with the latest version indicated in the badges above)."}
{"id":11572,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"The Java SDK uses slf4j for logging purposes. You can include the org.clojure\/tools.logging utility as a simple binding mechanism to send any logging information out to your console (standard out)."}
{"id":11573,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets."}
{"id":11574,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets."}
{"id":11575,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets."}
{"id":11576,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases."}
{"id":11577,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers."}
{"id":11578,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above."}
{"id":11579,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException. For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection."}
{"id":11580,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place."}
{"id":11581,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionException will be raised."}
{"id":11582,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":11583,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems. The SDK uses slf4j to be logging framework-agnostic."}
{"id":11584,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"You can therefore configure your own preferred logging framework: Add log4j2 bindings. The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException."}
{"id":11585,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 500 response indicating that something went wrong on the server side."}
{"id":11586,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"If any request encounters one of these problems, it will be retried. Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s."}
{"id":11587,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"(Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted. (This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client."}
{"id":11588,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out. By default, this is set to 80 seconds."}
{"id":11589,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client. Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds."}
{"id":11590,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds). Since version 0.9.0, the Java SDK supports connecting to multiple tenants."}
{"id":11591,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant. You can also (optionally) provide a second argument to directly give the API token for the tenant."}
{"id":11592,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Create an object as usual. You can access the operations for assets directly on the client, under client.assets."}
{"id":11593,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides. Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request."}
{"id":11594,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request. Alternatively, you can pass the client to the operation on the object itself."}
{"id":11595,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume."}
{"id":11596,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java."}
{"id":11597,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"command. These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object."}
{"id":11598,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Provide your username as the first argument. ."}
{"id":11599,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"and your password as the second argument, as a char[]. (Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) You need to create a built-in Java PasswordAuthentication object."}
{"id":11600,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Provide your username as the first argument. ."}
{"id":11601,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"and your password as the second argument, as a char[]. (Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":11602,"url":"https:\/\/developer.atlan.com\/sdks\/clojure\/","title":"Clojure SDK - Developer","text":"Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute."}
{"id":11603,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Obtain the SDK The Go SDK is currently in a pre-release, experimental state. While in this state, we reserve the right to make any changes to it (including breaking changes) without worrying about backwards compatibility, semantic versioning, and so on."}
{"id":11604,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"If you are eager to experiment with it, it is available on GitHub. You can use Go dependencies to install it directly from there."}
{"id":11605,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"We welcome your feedback during the pre-release, but cannot commit to any specific revisions or timelines at this point in time. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets!"}
{"id":11606,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":") ATLAN_BASE_URL should be given your Atlan URL (for example, https:\/\/tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system."}
{"id":11607,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"But it's your choice exactly how you manage the API token and including it for use within the client. That's it — once these are set you can start using your SDK to make live calls against your Atlan instance!"}
{"id":11608,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11609,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11610,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11611,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11612,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11613,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError."}
{"id":11614,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection."}
{"id":11615,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted."}
{"id":11616,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"In these cases, an ApiConnectionError will be raised. Atlan is a distributed, cloud-native application, where network problems can arise."}
{"id":11617,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses the slog library internally to provide a flexible framework for emitting log messages."}
{"id":11618,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using .SetLogger() on the context object with various logging levels: \"debug\" : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). \"info\" : used to confirm that things are working as expected."}
{"id":11619,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"\"warn\" : used as an indication that something unexpected happened, or as a warning of some problem in the near future. \"error\" : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":11620,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"You can enable logging by using .SetLogger() on the context object with various logging levels: \"debug\" : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). \"info\" : used to confirm that things are working as expected."}
{"id":11621,"url":"https:\/\/developer.atlan.com\/sdks\/go\/","title":"Go SDK - Developer","text":"\"warn\" : used as an indication that something unexpected happened, or as a warning of some problem in the near future. \"error\" : indicates that due to a more serious problem, the SDK has not been able to perform some operation."}
{"id":11622,"url":"https:\/\/developer.atlan.com\/sdks\/events\/","title":"Events - Developer","text":"Atlan produces events when certain activities occur in the system. You can tap into these in a push-based integration model to take action the moment they occur."}
{"id":11623,"url":"https:\/\/developer.atlan.com\/sdks\/events\/","title":"Events - Developer","text":"To tap into the events, you need to first set up a webhook in Atlan. Have a look at the event handling pattern for more details on implementing event-handling from webhooks using either of the SDKs."}
{"id":11624,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"Raw REST API Walk through step-by-step in our intro to custom integration course (30 mins). There are a number of details and nuances to understand about the underlying REST APIs to use them most effectively: the underlying REST API communications (HTTP protocols, methods, headers, endpoint URLs, response codes, etc) translating the complex JSON structures for each request and response payload knowing exactly which values are required (and which aren't) depending on the object you're interacting with, what operation you're carrying out, etc (including which exact string (and capitalization) to use for values that are really enumerations behind-the-scenes) Ultimately, all pull-based integration mechanisms (including the SDKs) use the REST API; however, the SDKs also encode best practices to avoid the need to understand all these details and low-level nuances."}
{"id":11625,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"If you want to adopt these best practices from the start, we would strongly recommend directly using any of our SDKs rather than the raw REST APIs directly: That being said, we have documented the raw REST API interactions in most cases. So if you really want to interact with the APIs directly, there should still be some guidance — anywhere you see Raw REST API gives details on endpoint URLs, methods, and payloads."}
{"id":11626,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"If all you really want to do is some initial experimentation directly against the API, you can use Postman. We do not maintain a Postman collection of requests, but you can use the Raw REST API tab to find the endpoint and an example payload to use for most operations here on developer.atlan.com."}
{"id":11627,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"Why not just publish an OpenAPI spec? We did try this in the past, as we liked the idea of generating client libraries using tools like the OpenAPI Generator."}
{"id":11628,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"Unfortunately, in our own testing of these tools, we found that: We could generate code that is free from syntax errors, but the generated code was not fully functional. Problems we found: The generated code drops significant portions of data from payloads."}
{"id":11629,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"(Our APIs are payload-rich and endpoint-light, while the generators seem to favor endpoint-rich and payload-light APIs.) The various objects the generator creates often make developer consumption cumbersome — long, difficult-to-read object names; many variations of similar objects; and so on. The generated code naturally does not include any validation that can't be encoded in the OpenAPI spec."}
{"id":11630,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"To add this we'd need to wrap the generated code with another layer anyway. After several attempts at mangling the OpenAPI spec to meet the constraints of the code generator, we eventually decided to go the way we've seen other API-first companies adopt."}
{"id":11631,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"We found very few API-first companies appear to rely on these code generators, but rather directly maintain their own SDKs for which they may have their own code generators. (Which is in fact exactly what we're doing as well.) Request for feedback If you use the raw REST APIs rather than one of the provided SDKs, we would love to understand more."}
{"id":11632,"url":"https:\/\/developer.atlan.com\/sdks\/raw\/","title":"Raw REST API - Developer","text":"If you can spare a few minutes to fill out our survey , we would be very grateful!"}
{"id":11633,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11634,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11635,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11636,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11637,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11638,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11639,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11640,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11641,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11642,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11643,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11644,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#__docusaurus_skipToContent_fallback","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11645,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11646,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11647,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11648,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11649,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11650,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11651,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11652,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11653,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11654,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11655,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11656,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#service-level-commitment","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11657,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11658,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11659,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11660,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11661,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11662,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11663,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11664,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11665,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11666,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11667,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11668,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#ways-to-contact-support","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11669,"url":"https:\/\/docs.atlan.com\/cdn-cgi\/l\/email-protection#721301193213061e131c5c111d1f","title":"Email Protection | Cloudflare","text":"You are unable to access this email address atlan.com The website from which you got to this page is protected by Cloudflare. Email addresses on that page have been hidden in order to keep them from being accessed by malicious bots."}
{"id":11670,"url":"https:\/\/docs.atlan.com\/cdn-cgi\/l\/email-protection#721301193213061e131c5c111d1f","title":"Email Protection | Cloudflare","text":"You must enable Javascript in your browser in order to decode the e-mail address. If you have a website and are interested in protecting it in a similar way, you can sign up for Cloudflare."}
{"id":11671,"url":"https:\/\/docs.atlan.com\/cdn-cgi\/l\/email-protection#721301193213061e131c5c111d1f","title":"Email Protection | Cloudflare","text":"How does Cloudflare protect email addresses on website from spammers? Can I sign up for Cloudflare?"}
{"id":11672,"url":"https:\/\/docs.atlan.com\/cdn-cgi\/l\/email-protection#721301193213061e131c5c111d1f","title":"Email Protection | Cloudflare","text":"Cloudflare Ray ID: 97e29b39d988990a • Your IP: Click to reveal 130.91.58.216 • Performance & security by Cloudflare"}
{"id":11673,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11674,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11675,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11676,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11677,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11678,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11679,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11680,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11681,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11682,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11683,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11684,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#hours-of-operation","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11685,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11686,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11687,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11688,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11689,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11690,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11691,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11692,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11693,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11694,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11695,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11696,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#severity-levels","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11697,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"One of Atlan's core values is to help you and your team do your life's best work. ð That's why Atlan wants to make it as easy as possible for you to keep driving your work forward with data."}
{"id":11698,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Atlan's customer support is a combination of several teams in Atlan: Product support personnel Cloud support personnel DevOps\/engineering support personnel Vast repository of self-service resources Service-level commitment Atlan's Technical Support team provides support globally with high response commitment levels. This includes 24\/7 SRE support for critical (P0) issues."}
{"id":11699,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Customers get a service-level commitment, including the following: 99.5% uptime for Atlan Dedicated support center, available from within the Atlan product Commitments for aggressive response times for business critical issues Designated Customer Success Manager to assist with escalations Ways to contact support ï¸ Email support at a dedicated customer support email account ( [email protected] ) ð ¨ ð » In-product support widget to log tickets and a help desk portal to log and track tickets. You can sign up to track support tickets on the help desk portal."}
{"id":11700,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"You must use your organizational email address as the username and create a password. ð Submit a support request via the online form."}
{"id":11701,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"To track your support tickets: Navigate to https:\/\/atlan.zendesk.com and log into the help desk portal with your credentials or via SSO. From the top right, click your avatar, and then from the dropdown, click My activities."}
{"id":11702,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"On the My activities page, you can do the following: My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11703,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11704,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"My requests and Requests I'm CC'd on - view and edit the support tickets you either created or were copied on, respectively. Organization requests - to access all other support tickets for your organization, please reach out to your customer success manager."}
{"id":11705,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Atlan provides you with read access to all the support tickets for your organization. To be able to comment on or close them, you must be CC'd on all tickets."}
{"id":11706,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Hours of operation 24x7 availability for all requests and issues Severity levels The Atlan Technical Support team determines the severity of an issue. The customer's position is considered, and these guidelines are followed to determine priority."}
{"id":11707,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"Below are the response time SLAs: Escalation procedure If the business impact of a support request changes or a ticket isn't being handled according to your expectations, you may escalate the ticket. Please first speak with the Technical Support representative assigned to the ticket to confirm that the business impact and urgency are understood."}
{"id":11708,"url":"https:\/\/docs.atlan.com\/support\/references\/customer-support#escalation-procedure","title":"Customer support | Atlan Documentation","text":"You may further escalate by contacting: 1st level of escalation : Technical Support Engineer 2nd level of escalation : Director, Support 3rd level of escalation : Head of Customer Experience Ways to contact support Hours of operation"}
{"id":11709,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Overall site map If you are new to Atlan, or to developing with Atlan, start with one of the following two options. These will set you up to develop with Atlan, step-by-step."}
{"id":11710,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Atlan University Self-paced, video-based walkthrough of the essentials of Atlan as a platform. Atlan Platform Essentials certification Self-paced, video-based walkthrough of the essentials of Atlan as a platform."}
{"id":11711,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Atlan Platform Essentials certification Introductory walkthrough Documentation-based walkthrough, including step-by-step examples. Start the walkthrough Documentation-based walkthrough, including step-by-step examples."}
{"id":11712,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Start the walkthrough Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11713,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11714,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11715,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11716,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11717,"url":"https:\/\/developer.atlan.com\/concepts\/#overall-site-map","title":"Developing with Atlan - Developer","text":"Learn more about events"}
{"id":11718,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Overall site map If you are new to Atlan, or to developing with Atlan, start with one of the following two options. These will set you up to develop with Atlan, step-by-step."}
{"id":11719,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Atlan University Self-paced, video-based walkthrough of the essentials of Atlan as a platform. Atlan Platform Essentials certification Self-paced, video-based walkthrough of the essentials of Atlan as a platform."}
{"id":11720,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Atlan Platform Essentials certification Introductory walkthrough Documentation-based walkthrough, including step-by-step examples. Start the walkthrough Documentation-based walkthrough, including step-by-step examples."}
{"id":11721,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Start the walkthrough Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11722,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11723,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11724,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11725,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11726,"url":"https:\/\/developer.atlan.com\/concepts\/#introduction","title":"Developing with Atlan - Developer","text":"Learn more about events"}
{"id":11727,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Overall site map If you are new to Atlan, or to developing with Atlan, start with one of the following two options. These will set you up to develop with Atlan, step-by-step."}
{"id":11728,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Atlan University Self-paced, video-based walkthrough of the essentials of Atlan as a platform. Atlan Platform Essentials certification Self-paced, video-based walkthrough of the essentials of Atlan as a platform."}
{"id":11729,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Atlan Platform Essentials certification Introductory walkthrough Documentation-based walkthrough, including step-by-step examples. Start the walkthrough Documentation-based walkthrough, including step-by-step examples."}
{"id":11730,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Start the walkthrough Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets."}
{"id":11731,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets."}
{"id":11732,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets."}
{"id":11733,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases."}
{"id":11734,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers."}
{"id":11735,"url":"https:\/\/developer.atlan.com\/concepts\/#jumping-in","title":"Developing with Atlan - Developer","text":"Learn more about events"}