Learn how to schedule and orchestrate SAS programs, flows, and jobs using Apache Airflow.
Learn How To
Schedule and orchestrate SAS programs, flows, and jobs using Apache Airflow.Use Airflow sensors, time, SQL, and dataset dependencies to trigger workloads.Use the SAS Airflow operator, macro variables, etc.Who Should Attend
The workshop would be suitable for anyone with a basic understanding of SAS Viya and an interest in learning how to use Apache Airflow to create more efficient and reliable data workflows. It may also be beneficial for those looking to modernize their data processing operations by integrating SAS Viya with contemporary workflow management tools. The list may include SAS Programmers, Data Engineers, Data Scientists/Analysts, Business Intelligence Professionals, DevOps Engineers, IT Professionals, System Administrators, Technical Managers/Project Managers.
Prerequisites
Before attending this course, you should be familiar with:;
Basic knowledge of SAS Viya: Understanding of SAS programming language and environment, as well as familiarity with SAS Viya's capabilities.Programming Skills: Proficiency in at least one programming language that is commonly used in data environments, such as Python, which is often used in conjunction with Apache Airflow.Workflow Automation: An understanding of the importance of automating repetitive tasks and how it applies to data workflows.Familiarity with DevOps Principles: For those who will be integrating SAS Viya with Apache Airflow into a DevOps workflow, a basic understanding of DevOps practices could be helpful.SAS Products Covered
SAS Studio
Course Outline
Airflow Basics
Create a basic DAG, which stands for Directed Acyclic Graph.SAS Studio OperatorCreate an Airflow DAG (Directed Acyclic Graph) using the SASStudioOperator, which involves importing the operator, creating a task, and setting various parameters such as path type, exec type, and connection name.Create DAGs to run SAS Studio flows.Create DAGs to run SAS Studio programs.Create DAGs to run SAS Studio jobs.The steps to explore and manage the updated DAG in Airflow, including triggering the DAG, checking task logs, and ensuring successful task execution.Time DependenciesTrigger a DAG containing SAS workloads using time dependencies or cron schedules.File DependenciesTrigger a DAG containing SAS workloads using file dependencies, for example whena file arrives in a folder.SQL DependenciesTrigger a DAG containing SAS workloads using SQL dependencies: when a certain criteria is met in a table's data.Macro VariablesPassing macro variables to SAS programs and downstream tasks from one SAS task to another in a DAG. Using the XCOM mechanism to push or pull variables or reusing the SAS Studio Compute session.