Add certificates | Atlan Documentation

How many times has someone complained to you that the data is incomplete or has issues? And how many times have you responded that it's still a work in progress or they're using the wrong data! Wouldn't it be really convenient if the data could answer these questions? Certificates in Atlan can help! The certification tags help users quickly identify whether a data asset is ready to use, a work in progress, or has some issues. You can add the following four certification tags to any data asset: Verified for ready to be used. Draft for work in progress. Deprecated if the asset no longer exists. No certificate if not required or needs documentation down the line. Certificates can be used to quickly filter data assets on the Assets page. This helps build trust in your data assets among users. Add certificates to your assets To add or update the certificate for your data assets, follow these steps: On the Atlan homepage, click Assets in the left menu. Click on the asset to open its asset profile. In the right menu, click + under Certificate and choose the relevant certification option. Write a message to add more context. Now your data can proudly display its status for all to see! ð Once you have selected a certification tag for your data asset, you will get a popup that your certificate has been saved. Add certificates to your assets


Link terms to assets | Atlan Documentation

Provide additional context for your assets to other users in your organization. Create common definitions once and apply them many times to multiple assets. Offer an abstract point for applying tags to be propagated to all linked assets - including their downstream and child assets - if propagation is enabled . If your data assets include personal information - for example, email addresses - you can link your assets to an Email Address term to provide context to your users. You can define the term Email Address once in the glossary. You can link the term to all the columns where an individual's email address appears. You can also tag the term as PII - and all of the linked assets will be tagged as PII . To link a term to an asset: From a term From the left menu on any screen, click Glossary . Under Glossary in the left menu, click the name of your glossary. Under your glossary name, click the category in which your term is nested and then click the term you would like to link to your assets. Click + Link Assets to get started. In the sidebar on the right, select the asset(s) to which you would like to link the term. At the bottom of the sidebar, click Link asset(s) to confirm your selections. From an asset From the left menu on any screen, click Assets . On the Assets page, select the asset to which you would like to link a term. In the dialog, expand the glossary menu and then click the term you would like to link to your assets. Click Save to confirm your selections. You can now view linked assets for your glossary term! ð To unlink a term from an asset: From a term From the left menu on any screen, click Glossary . Under Glossary in the left menu, click the name of your glossary. Under your glossary name, click the category in which your term is nested and then click the term you would like to unlink from your assets. Under Linked assets , navigate to the asset(s) from which you would like to unlink the term. To the right of the asset name, click the three dots and then click Unlink asset . From an asset From the left menu on any screen, click Assets . On the Assets page, select the asset from which you would like to unlink a term. Your assets will now be unlinked from the glossary term.


Search and discover assets | Atlan Documentation

Atlan is a living catalog of all your data assets and knowledge. It lets you quickly discover and access your data, along with the tribal knowledge and business context. Certify your assets Enrich your assets with descriptions Star your assets for easy access Intelligent keyword recognition Click Assets in the left-side panel. Domains : Filter assets by domains , such as a single domain, multiple domains, or no domain. Owners : Filter by selecting one or more users. You can also toggle between users and groups to filter based on a group of users. Tags : Filter by user-generated tags, such as public , PII , and more. Name : Sort by the asset name in an alphabetical or a reverse alphabetical order. Updated on Atlan : Sort by the newest or oldest updated assets. Star count : Sort assets by most or fewest stars . Popularity : Sort Snowflake and Google BigQuery assets by the most or least popular assets . The sorting options may vary depending on the asset type selected. For example, if you are viewing the results while filtering by the Table tab, you'll also have the option of sorting by the most or fewest number of rows and columns. See only what you want to see


Administrators | Atlan Documentation

User management User management is a critical part of data governance. Atlan's user management capabilities should be a mainstay of how you organize and control access for people in your organization. Add and manage users from the admin center It's super simple to invite and remove users from Atlan from the Admin center . You can also manage existing users by adding them to groups, changing their roles, or set up SSO , SCIM , and SMTP configurations. Manage access control from the governance center The Governance center is where you can build access control mechanisms to manage user access . Personas allow you to group users into teams, such as Financial Analysts or Cloud Engineers , and set policies based on the access those personas should have. Purposes are where you can build policies based on the actions or access that a user might need. For example, you can use Atlan's policy-based access controls to manage access to PII and other sensitive data. This is a best practice for data governance. Once you set these policies, Atlan will enforce them throughout your users' experience. This means that users who don't have access to a particular type of data will not be able to see it. Governance workflows help you set up robust controls on data access management, metadata enrichment, new entity creation, and more, with out-of-the-box workflow templates and automated execution. Asset profile The asset profile in Atlan gives you a quick and clear understanding of what a data asset contains. You can think of the asset profile as the TL;DR about your data. The glossary provides key intel on your data assets so you can quickly understand important attributes of your data, such as: Owners of your data, so you know who to ask for clarification. Certificate status, to easily understand if metadata enrichment is still in progress or the asset is ready to be used. Linked assets that are relevant to the term, so you can explore other helpful material. Here are a few of the things that make Atlan's discovery awesome: Intelligent keyword recognition sees through your typos to show exactly what you wanted, no matter what you actually typed. Sort by popularity to quickly discover what assets your teammates are using every day.


Secure Agent | Atlan Documentation

The Atlan Secure Agent is a lightweight, Kubernetes-based application that enables secure metadata extraction. It connects internal systems with Atlan SaaS while keeping sensitive data protected and doesn t require inbound connectivity. Running within an organization s controlled environment, the Secure Agent ensures compliance with security policies and automates metadata processing. Figure 1: The Secure Agent runs in the customer environment and acts as a gateway. Key capabilities The Secure Agent is designed for secure, scalable, and efficient metadata extraction. Security-first architecture Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary. Uses outbound, encrypted communication to interact with Atlan SaaS. Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance. Scalable metadata extraction A single deployment of the Agent can connect to multiple source systems. Supports multiple concurrent metadata extraction jobs. Uses Kubernetes-based workloads for efficient resource management. Flexible deployment Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) or on-premises clusters. Scales dynamically based on workload demands. Automated operations Continuously monitors system health and sends heartbeats to Atlan. Captures and uploads execution logs for troubleshooting and auditing. Provides performance insights through metrics and alerts. How it works The Secure Agent follows a job-based execution model where metadata extraction tasks are scheduled and executed within the organization's environment. The workflow typically involves: Atlan triggers a metadata extraction job. The Secure Agent retrieves job details and extracts metadata using source-specific connectors. Extracted metadata is shared with Atlan either through cloud storage or direct ingestion. Atlan workflows process the extracted metadata and publish the assets. Logs and execution status are sent to Atlan for monitoring and auditing. See also Deployment architecture : Learn more about how the Secure Agent integrates with your environment and supports secure metadata extraction. How it works


Secure Agent | Atlan Documentation


Deployment architecture | Atlan Documentation

The Atlan Secure Agent is a Kubernetes-based application that runs within a customer's environment. It acts as a gateway between the single-tenant Atlan SaaS and external systems like Snowflake, Tableau, and other data sources. This document explains the Secure Agent's deployment architecture, key components, communication flows, and security considerations. High-level architecture This section describes how the Secure Agent is structured and deployed. It explains the core components that enable metadata extraction, job execution, and communication with Atlan. Figure 1: Atlan Secure Agent deployment architecture. Core components The Secure Agent runs as a Kubernetes-based application within a customer's private cloud or on-premises environment. It consists of several key components that work together to execute metadata extraction tasks. Argo Workflows An Argo Workflow server is deployed to coordinate all activities and launch Kubernetes workloads. The Secure Agent uses Argo Workflows to orchestrate and manage metadata extraction jobs. Each workflow represents a unit of work, such as extracting metadata from a source system. Agent orchestrator A scheduled job that runs every five minutes to check for jobs that need to be executed. It connects to the Atlan tenant, retrieves job details, and initiates workflows accordingly. Auxiliary services Additional services that support agent operations: Health monitoring service sends periodic heartbeats to Atlan to confirm the agent is active. Logging service uploads execution logs to Atlan for monitoring and debugging. Metadata extraction workflows Connector-specific jobs that extract metadata from source systems. Workflows run in isolated containers, ensuring security and reliability. Data flow The Secure Agent supports two modes of metadata transfer. Each mode determines how extracted metadata is delivered to Atlan. Bucket relay Metadata extraction in bucket relay mode stores metadata in enterprise-managed cloud storage before Atlan retrieves it. Figure 2: Data flow in bucket relay mode. The Secure Agent extracts metadata and writes it to a storage bucket in the customer s cloud environment (such as AWS S3, Azure Blob Storage, or Google Cloud Storage). This is managed by providing the agent write access to cloud storage. Atlan retrieves metadata from the storage bucket and processes it further. This is managed by providing Atlan read access to list and read files in cloud storage. This mode ensures the extracted data remains within the customer s infrastructure until Atlan explicitly fetches it. Customers can also use this data for auditing. Direct ingestion In direct ingestion mode, metadata is transferred directly from the Secure Agent to Atlan. Figure 3: Data flow in direct ingestion mode. The Secure Agent uses pre-signed URLs to upload metadata directly to Atlan. Some cloud storage providers that use pre-signed URLs include: The Secure Agent uses pre-signed URLs to upload metadata directly to Atlan. Some cloud storage providers that use pre-signed URLs include: A pre-signed URL grants temporary access to upload files without exposing long-term credentials. A pre-signed URL grants temporary access to upload files without exposing long-term credentials. Each URL has an expiration time, ensuring access is only available for a limited duration. Each URL has an expiration time, ensuring access is only available for a limited duration. See also Secure Agent - Security : Details on security mechanisms.


Playbooks | Atlan Documentation

Get started How to set up playbooks Playbook management How to manage playbooks : Monitor and maintain your playbook workflows. How to automate data profiling : Set up automated data quality checks. Troubleshooting playbooks : Solutions for common playbook issues.


Playbooks | Atlan Documentation


Set up playbooks | Atlan Documentation

ð ¤ Who can do this? You will need to be an admin user in Atlan to create playbooks. A common question that data teams often face is how to automate metadata at scale. Having started out as a data team ourselves, we know that automating repetitive tasks can help data teams maximize the value they provide to their organization. One way of doing so is through Atlan's playbooks! Playbooks help power metadata automation for your data assets in Atlan. You can create rule-based automations at scale and update metadata in bulk, helping you streamline your workflows. You can update the following asset metadata using playbooks: For example, imagine your organization needs to transfer ownership of several data assets. Instead of your data team manually updating the ownership of each and every asset, you can create a playbook to automate this process and update the metadata of your assets at scale. Playbook recommendations Before you begin, review some general guidelines on running playbooks in Atlan: Avoid running multiple playbooks simultaneously on the same set of assets. Allow one playbook run to be completed before proceeding with another operation on the same set of assets. Otherwise, you may experience performance issues and inconsistencies. Review and understand the depth of your asset lineage or hierarchy prior to enabling a tag propagation playbook. For assets with complex lineage, tag propagation may take longer to complete than the playbook runtime. You may want to review and judiciously select a list of assets that need to be tagged directly. For their child and/or downstream assets, Atlan recommends that you enable tag propagation . Create a playbook To create a playbook in Atlan: From the Filters menu on the left or the tabs along the top, apply any asset filters . Click Governance to navigate to the governance center. Under the Governance heading of the Governance center , click Playbooks . Click Create New to get started. Under the Governance heading of the Governance center , click Playbooks . Click Create New to get started. In the Create new playbook dialog box, enter the following details: For Name , enter a name for the task to be accomplished - for example, Update ownership . (Atlan recommends that the length of a playbook name must be no longer than 46 characters.) (Optional) For Description , enter a description. (Optional) Select an icon for your playbook. For Name , enter a name for the task to be accomplished - for example, Update ownership . (Atlan recommends that the length of a playbook name must be no longer than 46 characters.) (Optional) For Description , enter a description. (Optional) Select an icon for your playbook. Click Create to save your playbook. Set up rules as filters To set up rules as filters for your playbook: In the Build Rules page of your playbook, click Filters . For name, add a name to your filter. To set a matching condition for the filters, select Match all or Match any . Match all will logically AND the criteria, while Match any will logically OR the criteria. For this example, we'll click Connection and then select a Snowflake connection. (Optional) To further refine your asset selection: Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection. Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection. Click Connector to filter assets by supported connectors . Click Asset type to filter by specific asset types - for example, tables, columns, queries, glossaries, and more. Click Certificate to filter assets by certification status . Click Owners to filter assets by asset owners . Click Tags to filter assets by your tags in Atlan, including imported Snowflake and dbt tags. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. Click Domains to filter by specific domains or subdomains to bulk update all the assets included in those data domains or subdomains. Click Products to filter for data products by specific data domains or subdomains. Click Schema qualified Name to filter assets by the qualified name of a given schema. Click Database qualified Name to filter assets by the qualified name of a given database. Click dbt to filter assets by dbt-specific filters and then select a dbt Cloud or dbt Core filter. Click Properties to filter assets by common asset properties . Click Usage to filter assets by usage metrics . Click Monte Carlo to filter assets by Monte Carlo-specific filters . Click Soda to filter assets by Soda-specific filters . Click Table/View to filter tables or views by row count, column count, or size. Click Column to filter columns by column-specific filters , including parent asset type or name, data type, or column keys . Click Process to filter lineage processes by the SQL query. Click Query to filter assets by associated visual queries . Click Measure to filter Microsoft Power BI measures using the external measures filter. Select Starts With or Ends With to filter assets using the starting or ending sequence of values. Select Contains or Does not contain to find assets with or without specified values contained within the attribute. Select Pattern to filter assets using supported Elastic DSL regular expressions . Select Is empty to filter assets with null values. Select Belongs to or Doesn't belong to to filter data products by specific data domains or subdomains . For Values , select the relevant values. The values will vary depending on the selected attributes. (Optional) To add more filters, click Add filter and select Filter to add individual filters or Filter Group to nest more filters in a group. (Optional) To view all the assets that match your rules, in the Filters card, click View all for a preview. (Optional) To remove a playbook filter, to the right of any filter, click the three horizontal dots and then click Delete . (Optional) To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable . Click Enable to turn on any disabled filters. Select the actions To select the actions to be performed based on your rules: In the Build Rules page of your playbook, click Actions . Click Certificate to update the certification status of assets to Verified , Draft , Deprecated , or No certificate . Click Description to update the description of your assets. Click Owners to add, remove, or replace asset owners . In this example, we'll update the ownership of the assets. Click Tags to add tags to your assets or remove or replace them from tagged or propagated assets. Note that if there are multiple tag actions to be performed, Atlan will execute them in the following order: ADD , REMOVE , and then REPLACE . Click Domain to add your assets to a specific domain or subdomain or remove them from an existing linked domain or subdomain . Click any custom metadata structure and then select a custom metadata property to update or unlink it from your assets. For Select operator , select the relevant option. The operators will vary depending on the selected action. For Values , select the relevant option(s). The values will vary depending on the selected actions. (Optional) To add more actions, click Add Action . You can control tag propagation when adding tags as an action in playbooks. Tag propagation is disabled by default. If you enable tag propagation, you will also be able to configure how tags are propagated . Run the playbook If you'd like to continue working on your playbook, you can save it as a draft. If your playbook is ready, you can proceed to running it. To run the playbook: You can either: To run the playbook once immediately, click Run once . To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. danger If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs. For more about workflows in general, see workflow recommendations . You can either: To run the playbook once immediately, click Run once . To run the playbook once immediately, click Run once . To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. danger If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs. For more about workflows in general, see workflow recommendations . To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs. For more about workflows in general, see workflow recommendations . Click Complete to run the playbook. Click Complete to run the playbook. In the resulting screen, click Go to profile to view your playbook profile. In the resulting screen, click Go to profile to view your playbook profile. Once your playbook has completed its run, you will see the metadata updated for your assets! ð If you have any questions about setting up playbooks, head over here . Create a playbook Set up rules as filters Select the actions Run the playbook


Manage playbooks | Atlan Documentation

Once you've created a playbook , you can monitor, modify, or delete it at any time. You can also enable notifications to monitor your playbook runs directly in Slack or Microsoft Teams. Monitor a playbook To monitor your playbooks runs: When running a playbook immediately, you will be redirected to the monitoring page within 5 seconds. From the left menu of any screen in Atlan, click Governance . Under the Governance heading of the Governance center , click Playbooks . In the playbooks manager, select the playbook you'd like to view. A summary of the rules, actions, and updated assets. An activity log for recent runs and updates over time. The activity log in the playbooks manager can help you keep track of playbook runs, ranging from 24 hours to 30 days. Select any of the entries to navigate to the corresponding playbook. Modify a playbook To modify an existing playbook: From the left menu of any screen in Atlan, click Governance. Under the Governance heading of the Governance center , click Playbooks . In the playbooks manager, click the playbook you'd like to modify. On your playbook page: To edit the name and description of your playbook, hover over your playbook and click Edit . To modify the rules of your playbook, click Rules to make your changes and then click Update to save them. (Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule . To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable . Click Enable to turn on any disabled filters. To modify the schedule for your playbook, in the upper right of your screen: Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule. To edit the name and description of your playbook, hover over your playbook and click Edit . To modify the rules of your playbook, click Rules to make your changes and then click Update to save them. (Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule . (Optional) To add a new rule to an existing playbook, in the left menu for playbook rules, click + Add new Rule . To turn off a playbook filter, to the right of any filter, click the three horizontal dots and then click Disable . Click Enable to turn on any disabled filters. To modify the schedule for your playbook, in the upper right of your screen: Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule. Click Run Now to run it immediately. Click the pencil icon to modify or remove the schedule. Delete a playbook To delete an existing playbook: From the left menu of any screen in Atlan, click Governance. Under the Governance heading of the Governance center , click Playbooks . In the playbooks manager, hover over the playbook you'd like to delete and click Delete Playbook . Click Delete to confirm. Enable playbook notifications You can set up Slack or Microsoft Teams alerts for your playbook runs in Atlan. This can help you monitor your playbooks directly in Slack or Microsoft Teams. You can also choose to receive alerts for failed playbook runs only. Before you can enable notifications for playbooks, you will need to either: Integrate Slack and Atlan Integrate Microsoft Teams and Atlan To enable notifications for playbook runs: From the left menu of any screen in Atlan, click Governance . Under the Governance heading of the Governance center , click Playbooks . In the upper-right of the playbooks manager, under Activity , click the bell icon. In the Enable notifications popup: Click Setup now to integrate Slack or Microsoft Teams . If you have already integrated Slack or Microsoft Teams, click Enable . Click Setup now to integrate Slack or Microsoft Teams . If you have already integrated Slack or Microsoft Teams, click Enable . In the notifications setup dialog, configure the following: For Notifications channel , you can either: If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected. You can use the same channel to receive both workflow and playbook run alerts and skip to the next step. If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs. This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration. To select the type of notifications you want to receive, you can either: Click Both success and failure alerts to receive notifications for both successful and failed playbook runs. Click Failure alerts only to limit notifications to failed playbook runs only. Click Save to save your notification preferences. For Notifications channel , you can either: If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected. You can use the same channel to receive both workflow and playbook run alerts and skip to the next step. If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs. This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration. If you have already configured a Slack or Microsoft Teams channel to receive workflow alerts, that channel will be preselected. You can use the same channel to receive both workflow and playbook run alerts and skip to the next step. If you have not configured a workflow alerts channel or want to add a different one, enter the channel name to receive notifications for playbook runs. This channel will be displayed as the Playbooks alert channel in your Slack or Microsoft Teams integration. To select the type of notifications you want to receive, you can either: Click Both success and failure alerts to receive notifications for both successful and failed playbook runs. Click Failure alerts only to limit notifications to failed playbook runs only. Click Both success and failure alerts to receive notifications for both successful and failed playbook runs. Click Failure alerts only to limit notifications to failed playbook runs only. Click Save to save your notification preferences. (Optional) To disable notifications, from the notifications setup dialog, remove the playbook alerts channel configured for Slack or Microsoft Teams . You will now receive Slack or Microsoft Teams notifications for all your playbook runs in Atlan! ð The Atlan bot will share playbook run alerts, including details like run status, start time, run time, trigger type, last three runs, and more. Monitor a playbook Modify a playbook Delete a playbook Enable playbook notifications


Automate data profiling | Atlan Documentation

Available via the Data Quality Studio package ð ¤ Who can do this? You need to be an admin user in Atlan to create profiling playbooks. Monitoring and improving data quality is critical to building trust in your data assets. Atlan solves for this with profiling playbooks! Profiling playbooks help power data observability for your assets in Atlan. You can create profiling playbooks to scan your assets at scale, identify any issues or inconsistencies, and improve the data quality of your assets. Supported sources Atlan currently supports column profiling for the following connectors: Microsoft SQL Server Create a profiling playbook To create a profiling playbook: In the left menu in Atlan, click Governance . Under the Governance heading of the Governance center , click Playbooks . To the right of the Create New button, click the downward arrow and then select Profiling Playbook . In the Create new profiling playbook dialog, enter the following details: For Name , enter a name for the task to be accomplished - for example, Tables scan . (Atlan recommends that the length of a playbook name must be no longer than 46 characters.) For Connection , select a supported connection from the dropdown menu - in this example, we'll select a Google BigQuery connection development . (Optional) For Description , enter a description for your playbook. (Optional) Select an icon for your playbook. For Name , enter a name for the task to be accomplished - for example, Tables scan . (Atlan recommends that the length of a playbook name must be no longer than 46 characters.) For Connection , select a supported connection from the dropdown menu - in this example, we'll select a Google BigQuery connection development . (Optional) For Description , enter a description for your playbook. (Optional) Select an icon for your playbook. Click Create to save your playbook. Set up rules as filters The assets to be scanned are pre-filled based on your selected connection. To set up rules as filters for your profiling playbook: In the Build Rules page of your profiling playbook, click Filters . For the name field, add a name to your filter - for example, Profiling action . To set a matching condition for the filters, select Match all or Match any . Match all will logically AND the criteria, while Match any will logically OR the criteria. Click Connection to select a specific connection. Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection. Click All databases to filter by databases in a selected connection. Click All schemas to filter by schemas in a selected connection. Click Connector to filter assets by supported connectors . Click Asset type to filter by specific asset types - for example, tables, columns, queries, glossaries, and more. Click Certificate to filter assets by certification status . Click Owners to filter assets by asset owners . Click Tags to filter assets by your tags in Atlan, including imported Snowflake and dbt tags. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. (Optional) For Snowflake tags only, to the left of the checkbox, click Select value , and then from the Select tag value dialog, select any value(s) to filter assets by tag value. Click Schema qualified Name to filter assets by the qualified name of a given schema. Click Database qualified Name to filter assets by the qualified name of a given database. Click dbt to filter assets by dbt-specific filters and then select a dbt Cloud or dbt Core filter. Click Properties to filter assets by common asset properties . Click Usage to filter assets by usage metrics . Click Monte Carlo to filter assets by Monte Carlo-specific filters . Click Soda to filter assets by Soda-specific filters . Click Table/View to filter tables or views by row count, column count, or size. Click Column to filter columns by column-specific filters , including parent asset type or name, data type, or column keys . Click Process to filter lineage processes by the SQL query. Click Query to filter assets by associated visual queries . Click Measure to filter Microsoft Power BI measures using the external measures filter. Select Starts With or Ends With to filter assets using the starting or ending sequence of values. Select Contains or Does not contain to find assets with or without specified values contained within the attribute. Select Pattern to filter assets using supported Elastic DSL regular expressions . Select Is empty to filter assets with null values. For Values , select the relevant values. The values will vary depending on the selected attributes. (Optional) To add more filters, click Add filter and select Filter to add individual filters or Filter Group to nest more filters in a group. (Optional) To view all the assets that match your rules, in the Filters card, click View all for a preview. Confirm profiling actions Column profiling is currently only supported for number and text data types. The profiled column assets will be populated with preconfigured metrics. To select the actions to be performed based on your rules: The default profiling actions to be performed include: Base metrics : Distinct count - number of rows that contain distinct values, relative to the column. Missing count - number of rows that do not contain specific values. Numeric metrics : Minimum and maximum values - smallest and greatest values in a numeric column. Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column. Variance - calculated variance of values in a numeric column. String metrics : Average length - average length of string values in a column. Minimum and maximum length - minimum and maximum length of string values in a column. Base metrics : Distinct count - number of rows that contain distinct values, relative to the column. Missing count - number of rows that do not contain specific values. Distinct count - number of rows that contain distinct values, relative to the column. Missing count - number of rows that do not contain specific values. Numeric metrics : Minimum and maximum values - smallest and greatest values in a numeric column. Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column. Variance - calculated variance of values in a numeric column. Minimum and maximum values - smallest and greatest values in a numeric column. Average - calculated average of values in a numeric column. Standard deviation - calculated standard deviation of values in a numeric column. Variance - calculated variance of values in a numeric column. String metrics : Average length - average length of string values in a column. Minimum and maximum length - minimum and maximum length of string values in a column. Average length - average length of string values in a column. Minimum and maximum length - minimum and maximum length of string values in a column. Click Next to proceed to the next step. In the Optimize your Profiling query popup, the following message will appear: This Profiling playbook will query x rows across y assets. To avoid significant computing costs, review your applied filters before proceeding . Click Review filters to review your existing filters or click Continue anyway to proceed. Note that Atlan is working to support sampling functionality in the future. Run the playbook If you'd like to continue working on your playbook, you can save it as a draft. If your playbook is ready, you can proceed to running it. To run the playbook: You can either: To run the playbook once immediately, click Run once . To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. To run the playbook once immediately, click Run once . To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time. Click Complete to confirm your selections. In the resulting screen, click Go to profile to view your playbook profile. Once your playbook run is completed, you will see the data profile updated for your assets! ð View profiled assets To view the profiled assets for your playbook: In the sidebar to the right, profiled assets will be indicated with a bar graph icon. Click any profiled asset to proceed to viewing profiling data. From the table sidebar, click the Column tab to view column assets and then select any of the profiled columns. In the column sidebar to the right, click Profile to view profiling data for the selected column asset. Once you've created a profiling playbook, you can monitor, modify, or delete it at any time. Create a profiling playbook Set up rules as filters Confirm profiling actions Run the playbook View profiled assets


Troubleshooting playbooks | Atlan Documentation

ð ¤ Who can do this? You will need to be an admin user in Atlan to create playbooks. Here are a few things to know about setting up playbooks : Following are the known issues or limitations when using the domain action : If you do not have read permission on the assets you want to add to a domain, those assets will be removed from the playbook workflow during processing. If you do not have update permission on the assets you want to add to a domain, the playbook workflow will fail. However, some assets may still be linked to the domain before the failure occurs. We recommend building no more than a maximum of 20 rules per playbook. However, the total number of playbooks that can be run is still to be determined. From a technical standpoint, playbooks leverage the workflow infrastructure, which means there are no hard limits. Depending on the number of playbooks that need to be run, the infrastructure will have to be scaled accordingly. Yes. You need to have the permission to update assets in Atlan in order to run playbooks for updating them. If you do not have the permission to update an asset, you will be unable to update it using playbooks. Additionally, Atlan uses the permissions of the playbook creator in determining the assets to be updated and not that of the user who runs the playbook. Your user permissions are used to determine the bulk updates you can make to ensure that there is no adverse impact on assets beyond your scope of access. No, Atlan currently does not support automating asset update requests through playbooks. Currently, there is no button to undo asset updates. However, you can modify your existing playbooks . You can either turn off the filters or add new rules to reverse the updates. Currently, no. You can monitor your existing playbooks to view a high-level summary of asset updates from previous playbook runs. Observability of results is on the roadmap. Currently, no. However, you can set up Slack or Microsoft Teams alerts for your playbook runs in Atlan. If you encounter an offload node status is not supported error message, the playbook workflow may have exceeded the EtcD size limit. Playbooks use Argo workflow templates, which are stored as Kubernetes resources. This creates a limit to their size. To handle this error, Atlan recommends the following: Reduce the number of rules in your playbook Optimize filters for asset selection


Contracts | Atlan Documentation

Get started Follow these steps to implement contracts in Atlan: Create data contracts Add contract impact analysis in GitHub : Detailed instructions on adding contracts for impact analysis in GitHub.


Contracts | Atlan Documentation


Create data contracts | Atlan Documentation

A data contract is an agreement between a data producer and consumer that specifies requirements for generating and using high-quality, reliable data. As a powerful tool for data management, data contracts can help you standardize contractual obligations between data producers and consumers, organize your assets with embeddable contract metadata, and enforce them with data quality rules. In Atlan, you can directly add a data contract to supported assets and provide helpful context to your downstream users. For a data contract to help build trust in your assets, it should be: Templatized and easily comprehensible - use Atlan's YAML contract template to create standardized contracts and push to Atlan. Version-controlled - continuously validate and monitor your data contracts either in runtime or real-time. Embeddable - embed the contract as metadata for a supported asset. Enforceable - enforce your contracts with data quality rules. Extensible - identify new specifications, generate new versions, and then compare and contrast them. You can create webhooks for data contracts and receive notifications for when a contract is added or updated to a URL of your choice. Supported asset types You can create data contracts for the following asset types: Output port assets of data products Supported asset metadata Atlan maps the following asset metadata properties to it contract properties: Add a data contract to an asset Any non-guest user with edit access to an asset's metadata can create, deploy, and manage data contracts. This only includes admin and member users in Atlan. To add a data contract to an asset, you can either: Create a contract directly in Atlan from the Contracts tab of the asset profile. You can create and maintain data contracts as easily as editing a word document. Use Atlan CLI to import an existing contract from your local machine to Atlan directly or through a CI/CD pipeline. Atlan CLI is a command-line tool that you can download directly from Atlan to your local machine to create and push data contracts to Atlan. Once you have published the contract, you can also sync metadata from a contract to the governed asset in Atlan. Once created, you will be able to monitor and manage your data contracts in Atlan. To add a data contract: From the left menu of any screen in Atlan, click Assets . (Optional) From the Filters menu on the left, click Properties and then click Has contract . Click No to filter for assets without a contract. From the Assets page, select an asset to open the asset sidebar. In the Contract tab of the asset profile, you can either: Click Create contract to create a draft contract directly in Atlan based on asset metadata. Click Import contract to use Atlan CLI to import an existing contract from your local environment to Atlan. You will first need to install and connect Atlan CLI and then push the contract to Atlan. Refer to our developer documentation to complete the steps. Click Create contract to create a draft contract directly in Atlan based on asset metadata. Click Import contract to use Atlan CLI to import an existing contract from your local environment to Atlan. You will first need to install and connect Atlan CLI and then push the contract to Atlan. Refer to our developer documentation to complete the steps. (Optional) Click the Edit button to edit the contract. Congrats on adding a data contract in Atlan! ð View a data contract To view a data contract: From the left menu of any screen in Atlan, click Assets . From the Assets page, select an asset to open the asset sidebar. Click the Document icon to open a read-only, simplified view of your contract. Next to Published version , click the version dropdown to view the latest version of the contract. Select an older version and then click Compare with published version to compare them side by side. Click the Edit button to edit the contract. Click the clipboard icon to copy the YAML code. Under Timeline , view a timeline for the evolution of your contract. Supported asset types Supported asset metadata Add a data contract to an asset View a data contract


Add contract impact analysis in GitHub | Atlan Documentation

Impact analysis helps you identify how modifications to your data contracts might impact downstream processes, data quality, and overall business operations. This can help you analyze proposed changes and mitigate potential risks before implementation. If you have ever changed a data contract only to find out later that it broke a downstream table or dashboard, Atlan provides a GitHub Action to help you out. This action places Atlan's impact analysis right into your pull request. So, you can view the potential downstream impact of your changes before merging the pull request. Before running the action, you will need to create an Atlan API token . You will also need to assign a persona to the API token and add a metadata policy that provides the requisite permissions on assets for the Atlan action to work. For example, you can add the following permissions: Asset, such as a table - Read only Any downstream connections, such as Microsoft Power BI - Read only Asset, such as a table - Read only Any downstream connections, such as Microsoft Power BI - Read only You will need to configure the default GITHUB_TOKEN permissions. Grant Read and write permissions to the GITHUB_TOKEN in your repository to allow the atlan-action to seamlessly add or update comments on pull requests. Refer to GitHub documentation to learn more. Configure the action To set up the Atlan action in GitHub: Create repository secrets in your repository: ATLAN_INSTANCE_URL with the URL of your Atlan instance. ATLAN_API_TOKEN with the value of the API token. ATLAN_INSTANCE_URL with the URL of your Atlan instance. ATLAN_API_TOKEN with the value of the API token. Add the GitHub Action to your workflow: Create a workflow file in your repository - .github/workflows/atlan-action.yml . Add the following code to your workflow file: name : Atlan action on : pull_request : types : [ opened , edited , synchronize , reopened , closed ] jobs : get-downstream-impact : name : Get Downstream Assets runs-on : ubuntu - latest steps : - name : Checkout uses : actions/checkout@v4 - name : Run Action uses : atlanhq/atlan - action@v1 with : GITHUB_TOKEN : $ { { secrets.GITHUB_TOKEN } } ATLAN_INSTANCE_URL : $ { { secrets.ATLAN_INSTANCE_URL } } ATLAN_API_TOKEN : $ { { secrets.ATLAN_API_TOKEN } } ATLAN_CONFIG : .atlan/config.yaml Create a workflow file in your repository - .github/workflows/atlan-action.yml . Create a workflow file in your repository - .github/workflows/atlan-action.yml . Add the following code to your workflow file: name : Atlan action on : pull_request : types : [ opened , edited , synchronize , reopened , closed ] jobs : get-downstream-impact : name : Get Downstream Assets runs-on : ubuntu - latest steps : - name : Checkout uses : actions/checkout@v4 - name : Run Action uses : atlanhq/atlan - action@v1 with : GITHUB_TOKEN : $ { { secrets.GITHUB_TOKEN } } ATLAN_INSTANCE_URL : $ { { secrets.ATLAN_INSTANCE_URL } } ATLAN_API_TOKEN : $ { { secrets.ATLAN_API_TOKEN } } ATLAN_CONFIG : .atlan/config.yaml Add the following code to your workflow file: Test the action After you've completed the configuration above, create a pull request with a changed Atlan data contract file to test the action. You should see the Atlan GitHub action running and then adding comments in your pull request: The GitHub workflow will add and update a single comment for every file change. The impacted assets in the comment will be displayed in a collapsible section and grouped by source and asset type. View impacted assets directly in Atlan. Configure the action Test the action


Integrations | Atlan Documentation

Atlan integrates with a wide range of tools to help you automate workflows, connect with your favorite apps, and manage identity and access. These integrations connect your data catalog with the tools your teams already use, creating a seamless data experience across your tech stack. Key concepts Integration categories : Atlan offers integrations across five categories: project management, communication, collaboration, automation, and identity management. Connection methods : Most integrations use secure authentication methods like OAuth, API keys, or service accounts. Bi-directional sync : Updates flow between Atlan and integrated tools, ensuring data consistency across platforms. Custom webhooks : Extend Atlan's capabilities by building custom integrations using the provided APIs and webhooks. Core offerings Connect with platforms like AWS Lambda to automate data workflows and streamline routine tasks. Integrate with tools like Slack and Microsoft Teams to enhance team collaboration and knowledge sharing. Connect with SMTP for real-time alerts. Integrate with identity providers like Okta and Azure AD for seamless authentication and user management. Connect with tools like Jira and Service Now to link data assets to projects and track data-related tasks. Get started Select an integration Choose from Atlan's available integrations based on your team's tools and workflows. Follow the integration-specific setup guide to establish a secure connection with your tool. Test and activate Verify the integration is working correctly with a test action, then activate for your organization. Need a custom integration? Atlan provides APIs and webhooks that let you build custom integrations with any tool in your tech stack.


Integrations | Atlan Documentation


Introductory walkthrough - Developer

An introductory walkthrough You might also like our Atlan Platform Essentials certification . Not sure where to start? Allow us to introduce Atlan development through example. 1 We strongly recommend using one of our SDKs to simplify the development process. As a first step, set one up: The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out). Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters. Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter. You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e. for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging. The SDK is available on PyPI . You can use pip to install it as follows: Provide two values to create an Atlan client: Provide your Atlan tenant URL to the base_url parameter. (You can also do this through environment variables .) Provide your API token to the api_key parameter. (You can also do this through environment variables .) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging. The SDK is available on Maven Central , ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil. Provide two values to create an Atlan client: Provide your Atlan tenant URL as the first parameter. You can also read the value from an environment variable, if you leave out both parameters. Provide your API token as the second parameter. You can also read the value from another environment variable, by leaving out this parameter. You can then start writing some actual code to run within a static main method. (We'll show some examples of this further below.) Once the block is complete, any resources held by the client (i.e. for caching) will be automatically released. Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging. The SDK is available on GitHub , ready to be included in your project: Provide two values to set up connectivity to Atlan: Provide your Atlan tenant URL to the assets.Context() method. If you prefer using the value from an environment variable, you can use assets.NewContext() without any parameters. Provide your API token as the second parameter to the assets.Context() method. (Or again, have it picked up automatically by the assets.NewContext() method.) Set up logging for SDK You can also checkout to the advanced configuration section of the SDK to learn about how to set up logging. Don't forget to give permissions If you want to be able to access existing metadata with an API token, don't forget that you need to assign one or more personas to the API token that grant it access to metadata. Now that you have an SDK installed and configured, you are ready to code! Before we jump straight to code, though, let's first introduce some key concepts in Atlan: What is an asset? In Atlan, we refer to all objects that provide context to your data as assets . Each type of asset in Atlan has a set of: Properties , such as: Certificates Announcements Properties , such as: Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table Relationships to other assets, such as: Schema child tables Table parent schema Table child columns Column parent table In an object-oriented programming sense, think of an asset as an instance of a class. The structure of an asset (the class itself, in this analogy) is defined by something called a type definition , but that's for another day. So as you can see: There are many different kinds of assets: tables, columns, schemas, databases, business intelligence dashboards, reports, and so on. Assets inter-relate with each other: a table has a parent schema and child columns, a schema has a parent database and child tables, and so on. Different kinds of assets have some common properties (like certificates) and other properties that are unique to that kind of asset (like a columnCount that only exists on tables, not on schemas or databases). When you know the asset When you already know which asset you want to retrieve, you can read it from Atlan using one of its identifiers . We'll discuss these in more detail as part of updates, but for now you can think of them as: is a primary key for an asset: completely unique, but meaningless by itself is a business key for an asset: unique for a given kind of asset, and interpretable You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName . (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the asset.get_by_guid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the asset.get_by_qualified_name() method on the Atlan client, providing the type of asset you expect to retrieve and its qualified_name . (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the static get() method on any asset type, providing the client and either the asset's GUID or qualifiedName . (Each asset type is its own unique class in the SDK.) You can retrieve an asset using the assets.GetByGuid() method on the Atlan client, providing both the type of asset you expect to retrieve and its GUID. (Each asset type is its own unique class in the SDK.) You can also retrieve an asset using the assets.GetByQualifiedName() method on the Atlan client, providing the type of asset you expect to retrieve and its qualifiedName . (Each asset type is its own unique class in the SDK.) Note that the response is strongly typed: If you are retrieving a table, you will get a table back (as long as it exists). You do not need to figure out what properties or relationships exist on a table - the Table class defines them for for you already. In any modern IDE, this means you have type-ahead support for retrieving the properties and relationships from the table variable. You can also refer to the types reference in this portal for full details of every kind of asset. Retrieval by identifier can be more costly than you might expect Even though you are retrieving an asset by an identifier, this can be more costly than you might expect. Retrieving an asset in this way will: Retrieve all its properties and their values Retrieve all its relationships Imagine the asset you are retrieving has 100's or 1000's of these. If you only care about its certificate and any owners, you will be retrieving far more information than you need. When you need to find it first For example, imagine you want to find all tables named MY_TABLE : You can then run the request using Execute() . For example, if you want to know the certificate of the asset you only need to tack that onto the query: Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets. Only this line differs from the original query. You can chain as many include_on_results calls as you want to specify the properties and relationships you want to retrieve for matching assets. Only this line differs from the original query. You can chain as many includeOnResults calls as you want to specify the properties and relationships you want to retrieve for matching assets. Only this line differs from the original query. You can include as many attributes in IncludeOnResults as you want to specify the properties and relationships you want to retrieve for matching assets. Also gives the best performance If all you want to do is check or report on metadata, you should have a starting point from the information above. Or, now that you've found an asset of interest, maybe you want to update the asset with additional metadata ? Once again, before we jump to code, let's first understand some key concepts about how Atlan handles updates: Importance of identifiers Most operations on assets are upserts , that is, they could either create (insert) or update a given asset. How do you know which is going to happen? To answer this question, you need to understand how Atlan uniquely identifies each asset. Recall earlier we discussed asset's different identifiers in Atlan . Every asset in Atlan has at least the following two unique identifiers. These are both mandatory for every asset, so no asset can exist without these: Atlan uses globally-unique identifiers (GUIDs) to uniquely identify each asset, globally . They look something like this: As the name implies, GUIDs are: Globally unique (across all systems). Generated in a way that makes it nearly impossible for anything else to ever generate that same ID. 2 Note that this means the GUID itself is not : Meaningful or capable of being interpreted in any way Atlan uses qualifiedName s to uniquely identify assets based on their characteristics. They look something like this: Qualified names are not : Globally unique (across all systems). Instead, they are: Consistently constructed in a meaningful way, making it possible for them to be reconstructed. Note that this means the qualifiedName is: Meaningful and capable of being interpreted How these impact updates Since they are truly unique, operations that include a GUID will only update an asset, not create one. Conversely, operations that take a qualifiedName can: Create an asset, if no exactly-matching qualifiedName is found in Atlan. Update an asset, if an exact-match for the qualifiedName is found in Atlan. These operations also require a typeName , so that if creation does occur the correct type of asset is created. Unintended consequences of this behavior Be careful when using operations with only the qualifiedName . You may end up creating assets when you were only expecting them to be updated or to fail if they did not already exist. This is particularly true when you do not give the exact, case-sensitive qualifiedName of an asset. a/b/c/d is not the same as a/B/c/d when it comes to qualifiedName s. Perhaps this leaves you wondering: why have a qualifiedName at all? The qualifiedName 's purpose is to identify what is a unique asset. Many different tools might all have information about that asset. Having a common "identity" means that many different systems can each independently construct its identifier the same way. If a crawler gets table details from Snowflake it can upsert based on those identity characteristics in Atlan. The crawler will not create duplicate tables every time it runs. This gives idempotency. Looker knows the same identity characteristics for the Snowflake tables and columns. So if you get details from Looker about the tables it uses for reporting, you can link them together in lineage. (Looker can construct the same identifier for the table as Snowflake itself.) These characteristics are not possible using GUIDs alone. Limit to changes only Now that you understand the nuances of identifiers, let's look at how you can update metadata in Atlan. In general, you only need to send changes to Atlan. You do not need to send an entire asset each time you want to make changes to it. For example, imagine you want to mark a table as certified but do not want to change anything else (its name, description, owner details, and so on): You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName . Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset. This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want. In this example, we change the certificate status to VERIFIED . At the end of your chain of updates, you need to build the builder (into an object, in-memory). And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. You can update an asset without first looking the asset up, if you know (can construct) its identifying qualified_name . Using the updater() class method on any asset type, you pass in (typically) the qualified_name and name of the asset. You can then add onto the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED . And then, finally, you need to client.asset.save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName . Using the updater() static method on any asset type, you pass in (typically) the qualifiedName and name of the asset. This returns a builder onto which you can then chain any updates. You can then chain onto the returned builder as many updates as you want. In this example, we change the certificate status to VERIFIED . At the end of your chain of updates, you need to build the builder (into an object, in-memory). And then, finally, you need to .save() that object to persist those changes in Atlan (passing the client for the tenant you want to save it in). The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. You can update an asset without first looking the asset up, if you know (can construct) its identifying qualifiedName . Using the Updater() method on any asset type, you pass in (typically) the qualifiedName and name of the asset. This returns an object into which you can then place any updates. You can place into the returned object as many updates as you want. In this example, we change the certificate status to VERIFIED . And then, finally, you need to .Save() that object to persist those changes in Atlan. The response will contain details of the change: whether the asset was created, updated, or nothing happened because the asset already had those changes. Atlan will handle idempotency By sending only the changes you want to apply, Atlan can make idempotent updates. Atlan will only attempt to update the asset with the changes you send. Atlan leaves any existing metadata on the asset as-is. If the asset already has the metadata values you are sending, Atlan does nothing. It will not even update audit details like the last update timestamp, and is thus idempotent. What if you want to make changes to many assets, as efficiently as possible? Start by initializing a batch. Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with). You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not "full". Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch. Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with). You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example). You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not "full". Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch. Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with). You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not "full". Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Start by initializing a batch. Through this batch, we can automatically queue up and bulk-upsert assets — in this example, 20 at a time. Be sure to include any details you might need to make a decision about whether to update the asset or not (and what to update it with). You can then add() any updated objects directly into the batch you created earlier. The batch itself will handle saving these to Atlan when a sufficient number have been queued up (20, in this example). You must flush() the batch outside of any loop where you've added assets into it. This ensures any final remaining elements in the batch are still sent to Atlan, even if the batch is not "full". Finally, from the batch you can retrieve the minimal details about any assets it created or updated. Where to go from here Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ Note that this is intentionally kept as simple as possible. The walkthrough is not intended to be exhaustive. Where possible, we have cross-referenced other detailed examples elsewhere in the site. ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩ There are orders of magnitude lower chances of GUIDs conflicting with each other than there are grains of sand on the planet. (And generating them does not rely on a central ID-assigning registry.) ↩


Introductory walkthrough - Developer


Java SDK - Developer

Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes. You can include logback as a simple binding mechanism to send any logging information out to your console (standard out), at INFO -level or above. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https://tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system. But it's your choice exactly how you manage the API token and including it for use within the client. (Note that you can also explicity provide only the tenant URL, and the constructor will look for only the API key through an environment variable.) That's it — once these are set you can start using your SDK to make live calls against your Atlan instance! 🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException . For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems. The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the ch.qos.logback:logback-classic:1.2.11 logback binding with log4j2 bindings. The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException . When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried. Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted. (This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out. By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client. Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds). Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant. You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual. You can access the operations for assets directly on the client, under client.assets . These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides. Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request. Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java . command. These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument. . and your password as the second argument, as a char[] . (Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute.


Python SDK - Developer

Walk through step-by-step in our intro to custom integration course (30 mins). Obtain the SDK The SDK is currently available on pypi . You can use pip to install it as follows: Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https://tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Careful not to expose your API token! We generally discourage including your API token directly in your code, in case you accidentally commit it into a (public) version control system. But it's your choice exactly how you manage the API token and including it for use within the client. In some scenarios, you may not want to expose the entire API token or manage environment variables. Instead, you can provide the GUID of the API token, and the SDK will internally fetch the actual access token. When to use this approach: Building apps that use the SDK where token security is a concern When you want to avoid exposing full API tokens in your configuration For containerized applications that need secure token management Before using this approach, ensure your Argo template is configured with CLIENT_ID and CLIENT_SECRET : Create client from token GUID : Use AtlanClient.from_token_guid() to create a client using the GUID of an API token. The SDK will automatically fetch the actual access token using the configured CLIENT_ID and CLIENT_SECRET . That's it — once these are set you can start using your SDK to make live calls against your Atlan instance! 🎉 Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanError exception, and all API operations can potentially raise AtlanError . For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionError , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionError will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise. The SDK therefore automatically attempts to handle ephemeral problems. The SDK uses logging module of the standard library that can provide a flexible framework for emitting log messages. You can enable logging for your SDK script by adding the following lines above your snippets: You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). logging.INFO : used to confirm that things are working as expected. logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future. logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation. logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now). You can enable logging by using basicConfig with various logging levels: logging.DEBUG : used to give detailed information, typically of interest only when diagnosing problems (mostly used level in SDK). logging.INFO : used to confirm that things are working as expected. logging.WARN : used as an indication that something unexpected happened, or as a warning of some problem in the near future. logging.ERROR : indicates that due to a more serious problem, the SDK has not been able to perform some operation. logging.CRITICAL : indicates a serious error, suggesting that the program itself may be unable to continue running (not used in SDK as of now). By default, logs will appear in your console. If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan/logging.conf') : this will generate logs according to the configuration defined in pyatlan/logging.conf and will generate two log files: /tmp/pyatlan.log : default log file. /tmp/pyatlan.json : log file in JSON format. By default, logs will appear in your console. If you want to use file logging, you can add the following line: logging.config.fileConfig('pyatlan/logging.conf') : this will generate logs according to the configuration defined in pyatlan/logging.conf and will generate two log files: /tmp/pyatlan.log : default log file. /tmp/pyatlan.json : log file in JSON format. logging.config.fileConfig('pyatlan/logging.conf') : this will generate logs according to the configuration defined in pyatlan/logging.conf and will generate two log files: /tmp/pyatlan.log : default log file. /tmp/pyatlan.json : log file in JSON format. The SDK handles automatically retrying your requests when it detects certain problems: When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 429 response indicating that the request rate limit has been exceeded, and you need to retry after some time. When there is a 50x response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried. Before each retry, the SDK will apply a delay using an exponential backoff. (Currently the values for the exponential backoff are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted. (This is set to 5 by default.) By default, the SDK AtlanClient() has the following timeout settings: read_timeout : 900.0 seconds ( 15 minutes) connect_timeout : 30.0 seconds If you need to override these defaults, you can do so as shown in the example below: Since version 1.0.0, the Python SDK supports connecting to multiple tenants.[^1] When you use the AtlanClient() method you are actually setting a default client. This default client will be used behind-the-scenes for any operations that need information specific to an Atlan tenant. When you want to override that default client you can create a new one and use the set_default_client() method to change it: The AtlanClient() method will return a client for the given base URL, creating a new client and setting this new client as the default client. If you want to switch between clients that you have already created, you can use Atlan.set_default_client() to change between them. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. (And since in the majority of use cases you only need access to a single tenant, this means you can most likely just rely on the default client and the fallback behavior.) Pyatlan uses the Requests library which supports proxy configuration via environment variables. Requests relies on the proxy configuration defined by standard environment variables http_proxy, https_proxy, no_proxy, and all_proxy. Uppercase variants of these variables are also supported. You can therefore set them to configure Pyatlan (only set the ones relevant to your needs): To use HTTP Basic Auth with your proxy, use the http://user:password@host/ syntax in any of the above configuration entries: Currently, the way this is implemented limits you to either avoiding multiple threads in your Python code (if you need to use multiple clients), or if you want to use multiple threads you should only use a single client. Asynchronous SDK operations To get started, you need to initialize an AsyncAtlanClient : Create an async client using the same configuration pattern as the synchronous client. Concurrent operations for improved performance The real power of async comes from running multiple operations concurrently . Instead of waiting for each operation to complete sequentially, you can execute them in parallel and reduce total execution time: Synchronous : Total time = operation₁ + operation₂ + . + operationₙ Asynchronous : Total time = max(operation₁, operation₂, . , operationₙ) When to use async Async is most beneficial when you have: Multiple independent operations that can run concurrently I/O-heavy workloads like API calls, database queries, or file operations Long-running operations where parallelization provides significant time savings For simple, single operations, the synchronous client may be more straightforward to use.


Python SDK - Developer


Kotlin SDK - Developer

Obtain the SDK For Kotlin, you can reuse the existing Java SDK as-is. It is available on Maven Central, ready to be included in your project: Include the latest version of the Java SDK in your project as a dependency. You can also give a specific version instead of the + , if you'd like. The Java SDK uses slf4j for logging purposes. You can include slf4j-simple as a simple binding mechanism to send any logging information out to your console (standard out), along with the kotlin-logging-jvm microutil. Configure the SDK There are two ways to configure the SDK: Using environment variables ATLAN_API_KEY should be given your Atlan API token , for authentication ( don't forget to assign one or more personas to the API token to give access to existing assets! ) ATLAN_BASE_URL should be given your Atlan URL (for example, https://tenant.atlan.com ) Here's an example of setting those environment variables: On client creation If you prefer to not use environment variables, you can do the following: Delve into more detailed examples: Common tasks Common operations on assets, that are available across all assets. Discover actions Common operations on assets, that are available across all assets. Asset-specific Operations that are specific to certain assets. Focus on a specific kind of asset Operations that are specific to certain assets. Focus on a specific kind of asset Governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Operations dealing with governance structures, rather than assets. Manage governance structures Samples Real code samples our customers use to solve particular use cases. Review live samples Real code samples our customers use to solve particular use cases. Review live samples Events Delve deep into the details of the events Atlan triggers. Learn more about events Delve deep into the details of the events Atlan triggers. Learn more about events The SDK defines checked exceptions for the following categories of error: A given API call could fail due to all of the errors above. So these all extend a generic AtlanException checked exception, and all API operations throw AtlanException . For example, when creating a connection there is an asynchronous process that grants permissions to the admins of that connection. So there can be a slight delay between creating the connection and being permitted to do any operations with the connection. During that delay, any attempt to interact with the connection will result in a PermissionException , even if your API token was used to create connection in the first place. Another example you may occasionally hit is some network issue that causes your connection to Atlan to be interrupted. In these cases, an ApiConnectionException will be raised. Don't worry, the SDK retries automatically While these are useful to know for detecting issues, the SDK automatically retries on such problems. Atlan is a distributed, cloud-native application, where network problems can arise. These advanced configuration options allow you to optimize how the SDK handles such ephemeral problems. The SDK uses slf4j to be logging framework-agnostic. You can therefore configure your own preferred logging framework: Replace the org.slf4j:slf4j-simple:2.0.7 binding with log4j2 bindings. The SDK handles automatically retrying your requests when it detects certain problems: When an ApiConnectionException occurs that is caused by an underlying ConnectException or SocketTimeoutException . When there is a 403 response indicating that permission for an operation is not (yet) available. When there is a 500 response indicating that something went wrong on the server side. If any request encounters one of these problems, it will be retried. Before each retry, the SDK will apply a delay using: An exponential backoff (starting from 500ms) A jitter (in the range of 75-100% of the backoff delay) Each retry will be at least 500ms, and at most 5s. (Currently these values are not configurable.) For each request that encounters any of these problems, only up to a maximum number of retries will be attempted. (This is set to 3 by default.) You can configure the maximum number of retries globally using setMaxNetworkRetries() on a client. Set this to an integer: The SDK will only wait so long for a response before assuming a network problem has occurred and the request should be timed out. By default, this is set to 80 seconds. You can configure the maximum time the SDK will wait before timing out a request using setReadTimeout() on a client. Set this to an integer giving the number of milliseconds before timing out: Remember this must be given in milliseconds. This example sets the timeout to 2 minutes (120 seconds * 1000 milliseconds). Since version 0.9.0, the Java SDK supports connecting to multiple tenants. From version 4.0.0 onwards you can create any number of clients against any number of different tenants, since every operation that interacts with a tenant now explicitly requires a client to be provided to it: Constructing a new client with a different tenant's URL is sufficient to create connectivity to that other tenant. You can also (optionally) provide a second argument to directly give the API token for the tenant. Create an object as usual. You can access the operations for assets directly on the client, under client.assets . These will generally give you the most flexibility — they can handle multiple objects at a time and allow overrides. Every operation on the client itself has a variant with an (optional) final argument through which you can override settings like retry limits or timeouts for this single request. You can use the from(client) factory method to initialize the request options with all the settings of your client, and then you only need to chain on those you want to override for this particular request. Alternatively, you can pass the client to the operation on the object itself. Limit the number of clients to those you must have Each client you create maintains its own independent copy of various caches. So the more clients you have, the more resources your code will consume. For this reason, we recommended limiting the number of clients you create to the bare minimum you require — ideally just a single client per tenant. Using a proxy To use the Java SDK with a proxy, you need to send in some additional parameters when running any java . command. These are described in detail in the Java documentation , but are summarized here for simplicity: socksProxyHost should be set to the hostname for your SOCKS proxy socksProxyPort should be set to the port for your SOCKS proxy (default being 1080) Providing credentials to the proxy In either case, if you need to authenticate to your proxy, you will need to wrap whatever code you want to run to set up these credentials using something like the following: You need to create a built-in Java PasswordAuthentication object. Provide your username as the first argument. . and your password as the second argument, as a char[] . (Of course, you should not hard-code your password in your code itself, but rather pull it from elsewhere.) Then use setProxyCredential() to pass this PasswordAuthentication object to the Atlan client, before any of the rest of the code will execute.