Lesson 01

Statistics You Need to Know for Machine Learning
Lesson 01, Section 1 Demo: Automated Machine Learning in SAS Studio

Lesson1

Demo: Automated Machine Learning in SAS Studio (Self-Paced)

In this demonstration, you access and examine the SAS Studio interface. You then load the course data in memory by using a pre-created SAS Studio flow. By running this flow, you establish a CAS session and then you connect your session to the location of pre-existing data files to load them into memory. Any data processing in SAS Viya requires the data to be loaded in memory first. Finally, you create a flow with the Automated Feature Engineering step to generate features from the PVA_Donors data that can be used as inputs to a predictive model.

Examining the SAS Studio Interface

  1. From the Windows taskbar, launch Google Chrome. When the browser opens, select SAS Landing from the bookmarks bar. The user ID and password should be prefilled, but if not, sign in as student using the password Metadata0. (Be sure to use an uppercase M in the password.) If prompted, select Yes for assumable groups.

    2. Note: You can also directly select the SAS Studio page from the bookmarks bar, if available.

  2. Click the Applications menu button (the three-by-three grid) in the upper left corner, expand ANALYTICS LIFE CYCLE, and select Develop Code and Flows to launch SAS Studio. The SAS Studio Develop Code and Flows window is displayed. The main window of SAS Studio consists of a navigation pane on the left and a work area on the right. The navigation pane provides easy access to your snippets, steps, and libraries, as well as other components. The work area is used to display your data, code, logs, results, and flows.
    3. An image of the Navigation Pane and Work Area.
  3. In the left navigation pane, click the Explorer icon (second from the top). Programs and Flows will be run from the VST folder (Files/Home/workshop/VST). Note: The path to the VST folder may change in later versions of SAS Viya.

  4. Next, click the Steps icon (third from the top). Steps can be used to create flows which automate data manipulation and analysis tasks.

  5. Now click the Libraries icon (second from bottom) to access the default SAS libraries. You see the WORK, SASHELP, and SASUSER libraries, as well as others. Next, you create and access custom caslibs.

Assigning Libraries and Loading Course Data

Next you run the pre-created SAS Studio flow to assign libraries and load course data. A flow consists of one or more objects. Each object is represented by a node in the process flow. The flow shows the relationship between two or more objects such as a SAS program, a task, a query, and so on. You access pre-created flow as follows:

  1. In the left navigation pane, click the Explorer icon (second from top). Expand Files > Home > workshop > VST > AssignLibrary.flw and double-click Assign Library flow.

    2. Note: The SAS Program node in this flow executes a SAS program behind the scenes to establish a CAS session, assign libraries, and load the data in memory. A discussion about the underlying code in the flow is beyond the scope of this course.

  2. Run the flow by clicking the Run toolbar button.

  3. A green check mark on the Assign Library node indicates that the flow has run successfully. Click the Libraries icon (second from bottom) in the left navigation pane. Look for the VST library and expand it to ensure that the PVA_DONORS table along with other data tables are loaded in memory and are accessible via the VST library. With this step completed, one of the important tasks of loading the data into memory has been accomplished. You analyze this PVA_DONORS data set in upcoming lessons. However, to start with, let's move directly into automated machine learning in SAS Viya.

    4. Note: A pre-made flow with steps for all the demonstrations in this course is located in the VST/backup flows folder.

    Optional: After initially running AssignLibrary.flw, this flow is used later to re-establish the VST library whenever SAS Studio times out. As an optional alternative approach, the code from this flow can be added to the Autoexec file. Code in the Autoexec file is executed whenever SAS studio is started. This will result in the VST library being established whenever the user logs into SAS studio. To do this, click on Options at the top left side of SAS Studio, then choose Autoexec file. Copy and paste the code from the AssignLibrary flow to the Autoexec window, then click Save. The VST library will now be available whenever you log into SAS Studio.

Automated Machine Learning

  1. In the work area, click the plus sign next to the AssignLibrary.flw tab to create a new tab. From the drop-down menu, choose Flow. Alternatively, select Flow from the New menu at the top left of SAS Studio.

  2. Click on the Steps icon on the left side of the Navigation Pane. Under SAS Steps tab, expand Data (Input and Output) group, double click on the Table step.

  3. In the work area, click on the Table step. At the lower half of the work area, under Table Properties tab, click the file cabinet next to the Library field. Under Libraries on the left, click on the VST library, then select the PVA_DONORS data on the right and click OK.

  4. Scroll down through SAS Steps and expand Machine Learning group. Double click on theAutomated Feature Engineering step. The Automated Feature Engineering step explores the variables in the input data and automatically performs feature engineering in a parallel, scalable way.

  5. Next, move your mouse towards the edge of the PVA_DONORS node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the Automated Feature Engineering node (the input port). Now Automated Feature Engineering is connected to the PVA_DONORS data. Click the Arrange nodes button to visually organize the nodes within the flow.
    6. An image of the Arrange Nodes button.
  6. Click on the Automated Feature Engineering node. in the lower half of the work area, under the Data tab, find the Target field. Make sure Use a nominal target radio button is selected. Use the plus sign to select a nominal target variable. Scroll down the variable list and click on Response, then choose OK.

  7. Under Inputs, ensure that Use all columns except target is selected.

  8. Click the Options tab. Keep the settings under Variable Screening, Feature Engineering, and Feature Ranking at their defaults.

  9. Click Run to execute the flow. Save the flow to the VST folder as Auto Feature Engineer.flw. When the run is complete, maximize the results window by clicking the three dots on the right side of the work area, choose Open in a browser tab, then choose Results.
    10. An image of the Label Column.

Engineered features have been created automatically using appropriate feature transformation and extraction techniques. The idea was to automatically learn a set of features (from potentially noisy, raw data) that can be useful in machine learning tasks without needing manually created, handcrafted engineered features.

In the results window, turn your focus to the Label column. What is so special about this Label column? This is the column that holds the description of the newly generated features.

Note: Other columns in the results window are not discussed here. However, they are covered in detail in other SAS courses.

Going forward, you might want to use these generated features for model building. However, the intention of this demo was to explore the automated machine learning capabilities of the tool.

Lesson 02

Statistics You Need to Know for Machine Learning
Lesson 02, Section 1 Demo: Listing Observations

TopicTitle

Demo: Listing Observations (Self-Paced)

In the previous lesson, you loaded all the course data into memory. In this demonstration, you use a SAS Studio flow to list the first 10 observations in the PVA_DONORS data set.

Reminder: If you restarted your SAS session, or it has timed out due to inactivity, you must re-create the course libraries, LOCALLIB and VST. To do this, run the AssignLibrary flow.

  1. In SAS Viya, launch SAS Studio by clicking the Applications menu button (the three-by-three grid in the upper left corner), expand ANALYTICS LIFE CYCLE, and click Develop Code and Flows.

  2. In the navigation panel on the far left, click the Explorer button. Expand Files >Home > workshop > VST. Double-click AssignLibrary.flw to open it.

  3. Click the Run button on the flow toolbar.

Demo Steps

  1. In the work area, click the plus sign next to the open tabs to create a new tab. From the drop-down menu, choose Flow. Alternatively, select Flow from the New menu at the top left of SAS Studio.

  2. Click on the Steps icon on the left side of the Navigation Pane. Under SAS Steps, Data (Input and Output), double click on Table.

  3. In the work area, click on the Table step. At the bottom, under Table Properties, click the file cabinet next to the Library field. Click on VST under Libraries, then select the PVA_DONORS data on the right and click OK.

  4. Under SAS Steps, expand Examine Data, then double click on List Data.

  5. Next, move your mouse towards the edge of the PVA_Donors node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the List Data node (the input port).Now List Data is connected to the PVA_Donors data. Click the Arrange nodes button to visually organize the nodes within the flow.

  6. Click on the List Data node, then click on theOptions tab. Scroll down to Rows to list field and use the dropdown arrow to select First n rows. Leave the default value of Number of rows set to 10 rows.

  7. Use the save button to save the flow to the VST folder as Explore PVA.flw. Click Run to run the flow. To view the results, click on Submitted Code and Results, then click Results.

    8. An image showing the Submitted Code and Results and Results.

Alternatively, click the three vertical dots on the upper right corner of the open tab, then choose Open in a browser tab, then Results, to open the results in a new browser tab.

An image showing the pathway to Results.
  • Use the double arrows in the upper right of the navigation pane to hide the navigation pane and maximize the results. The results appear in the Work area. To increase the size of the work area, locate the List Data step settings in the lower half of the work area. Click the Minimize preview button (a minus sign icon) at the top right corner of this section.
    • An image showing the Minimize Preview Button.
  • The successful submission of the List Data step produces a list of the first 10 observations from the PVA_DONORS data set. Listing data provides a quick snapshot of your analysis table. By default, the output of the List Data step shows variable labels. You can easily identify the gift-related inputs, such as Gift Count 36 Months, Gift Amount Last, and so on, that display the previous donation history. Scrolling to the right, you can see the demographic inputs, such as Demographic Cluster, Age, and Gender, that provide demographic characteristics about the donors. And last but not the least, there are the two target variables, Target Gift Flag (Response) and Target Gift Amount (Donation_Amt). The binary variable Target Gift Flag represents the response to the campaign and has two levels, Yes and No. The continuous variable Target Gift Amount represents the donation amount from the individuals who have responded to the campaign.

  • You will use both target variables in this course, but only one at a time.

Statistics You Need to Know for Machine Learning
Lesson 02, Section 2 Demo: Exploring Data

Demo: Exploring Data (Self-Paced)

This demonstration illustrates performing initial data exploration in three parts.

Reminder: If you restarted your SAS session, or it has timed out due to inactivity, you must re-create the course libraries,LOCALLIB and VST. To do this, run the AssignLibrary flow.

  1. In SAS Viya, launch SAS Studio by clicking the Applications menu button (the three-by-three grid in the upper left corner), expand ANALYTICS LIFE CYCLE, and click Develop Code and Flows.

  2. In the navigation panel on the far left, click the Explorer button (second from top). Expand Files >Home > workshop > VST. Double-click AssignLibrary.flw to open it.

  3. Click the Run button on the flow toolbar.

Part 1: Obtaining Summary Statistics for Continuous Variables

In this demonstration, you perform initial data exploration using a flow with the Summary Statistics node. You obtain summary statistics for continuous variables.

  1. Open the previously saved flow Explore PVA.flw from the VST folder.

  2. Under SAS Steps, expand the Statistics group, then double click on the Summary Statistics step.

  3. Next, move your mouse towards the edge of the PVA_Donors node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the Summary Statistics node (the input port). Now Summary Statistics node is connected to the PVA_Donors data. Click the Arrange nodes button to visually organize the nodes within the flow.
    4. An image of the PVA_Donors tab.
  4. Click on the Summary Statistics node, then select the Data tab. Next to Analysis variables, click the plus sign to add variables. Click the box next to Name to select all the variables. Click OK.

  5. Click on the Option tab. The Mean, Standard deviation, Minimum value, Maximum value, and Number of observations are selected by default. Keep these options and additionally select Number of missing values.

  6. Still under the options tab, expand Additional Statistics. Select Mode, Skewness, and Kurtosis. Expand Percentiles, then check off Lower quartile, Median, and Upper quartile.

  7. Save the flow then right-click on the Summary Statistics node and choose Run node. Open the Results in a new browser tab.

An image of the Results entries.

Part 2: Distribution Analysis of Continuous Variables

In this demonstration, you use a flow with the Distribution Analysis node to examine the distribution of continuous variables.

  1. Return to the previous flow containing the Summary Statistics step (Explore PVA.flw). From the Steps menu, under SAS Steps, expand Statistics, then double click on Distribution Analysis.

  2. Move your mouse towards the edge of the PVA_Donors node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the Distribution Analysis node (the input port). Now Distribution Analysis is connected to the PVA_Donors data. Click the Arrange nodes button to visually organize the nodes within the flow.
    3. An image of the PVA_Donors node.
  3. Click on the Distribution Analysis node, then select the Data tab. Next to Analysis variables, click the plus sign to add variables. Add all the variables except for StatusCatStarAll. A quick way of getting this done is by clicking the select all box next to Name and then deselecting StatusCatStarAll. Click OK to close the Column Selection window.

  4. Click the Options tab. Verify that Histogram has been selected. Select the Add normal curve box under Exploring Data to include a normal curve for the histogram. To include specific summary statistics, select the Add inset statistics option and expand the Inset Statistics properties and select Mean, Standard deviation, Skewness, and Kurtosis in addition to the default of Number of observations. To assess normality using the normal probability plot, select the Normal probability plot box under Checking for Normality.

  5. Right click on the Distribution Analysis node and choose Run node. Save the file Explore PVA.flw. When the run is complete, you can view the results on a browser tab. Click the More options on the right side of the screen (at the top right side of the open tab). Click Open in a browser tab and then click Results. View the results.

Note: If you see a warning message in the results window that says The maximum size limit has been exceeded, you might want to click the Open in new tab link to view the results. Alternatively, go to the SAS Studio Options menu, and select Preferences. Under SAS Programs, click Results. Under Display warning if results are larger than specific size, change the Maximum result size (MB) from 3 to 10.

Your results window displays histograms for all the analysis variables assigned in the Analysis variables property. In a practical scenario, it is worth spending time to examine the shape and spread of all the analysis variables in data. However, this is overwhelming in the class setup. So, let's examine the distribution of a few chosen variables. Consider the histogram for the Gift Count 36 Months variable.

An image of the Distribution of GiftCnt36.

The histogram and the normal curve superimposed on it indicate that the distribution is positively skewed, and the same can be ascertained from the statistics in the inset box. Scroll further down to examine the distribution of the DemMedIncome variable.

An image of the Distribution of DemMedIncome.

You clearly see a spike around the zero value, which accounts for approximately 25% of the income values. This resonates with the summary statistics, which indicate its mode value to be zero.

Now you consider examining the distribution of our interval-valued target variable: Donation_Amt (Target gift amount).

An image of the Distribution of Donation_Amt.

The histogram superimposed with the normal curve indicates that the variable is relatively normally distributed.

Next, to examine the normal probability plot for Donation_Amt, scroll down further until the bottom of page, and you see that the distribution of data points follows the normal reference line quite closely.

An image of the Probability Plot for Donation_Amt.

Thus, the data distribution looks fairly normal. This is an important insight about the interval valued target variable which could be useful in upcoming lessons on modeling.

Part 3: Generating Tables and Plots for Categorical Variables

In this demonstration, you use a flow with the One-Way Frequencies node to generate frequency tables and frequency plots from your data.

  1. Return to the previous flow Explore PVA.flw. From the Steps menu, under SAS Steps, expand Statistics, then double click on One-Way Frequencies.

  2. Next, move your mouse towards the edge of the PVA_Donors node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the One-Way Frequencies node (the input port). Now One-Way Frequencies is connected to the PVA_Donors data. Click the Arrange nodes button to visually organize the nodes within the flow.
    3. An image of the PVA_DONORS node.
  3. Click on the One-Way Frequencies node, then select the Data tab. Next to Analysis variables, click the plus sign to add variables. Add the variables StatusCat96NK, StatusCatStarAll, DemCluster, DemGender, DemHomeOwner, and Response variables.

  4. No changes are needed in the Options tab.

  5. Save the flow. Click right click on the One-Way Frequencies node and click Run node. Click on Submitted Code and Results, then click Results to view the report. Alternatively, click the More options on the right side of the screen (three vertical dots at the top right side of the open tab), then click Open in a browser tab, and finally click Results.

The flow displays the frequency table and bar charts for the requested variables. Note that the DemCluster variable has as many as 54 distinct levels. Do you see any challenges involving using a categorical variable that has as many as 54 levels as input for model building? Scroll further to examine the distribution of yet another target variable in our data set, Response. The frequency table and distribution plot clearly indicate that the data set has an equal proportion of responses and non-responses.

An image of the Target Gift Flag.

Do you see any relationship between proportion of non-responses (in the Response variable) and the number of missing values in Donation_Amt?

Statistics You Need to Know for Machine Learning
Lesson 02, Section 3 Demo: Testing a Hypothesis Using a One-Sample t-Test (Self-Paced)

Demo: Testing a Hypothesis Using a One-Sample t-Test (Self-Paced)

In this demonstration, you use a flow with the t-Tests node to test whether the population mean donation amount is $100 given the information in the PVA_DONORS table.

Reminder: If you restarted your SAS session, or it has timed out due to inactivity, you must re-create the course libraries, LOCALLIB and VST. To do this, run the AssignLibrary flow.

  1. In SAS Viya, launch SAS Studio by clicking the Applications menu button (the three-by-three grid in the upper left corner), expand ANALYTICS LIFE CYCLE, and click Develop Code and Flows.
  2. In the navigation panel on the far left, click the Explorer button (second from top). Expand Files >Home > workshop > VST. Double-click AssignLibrary.flw to open it.
  3. Click the Run button on the flow toolbar.

Demo Steps

  1. In the work area, click the plus sign next to the open tabs to create a new tab. From the drop-down menu, choose Flow. Alternatively, select Flow from the New menu at the top left of SAS Studio.
  2. Click on the Steps icon on the left side of the Navigation Pane. Under SAS Steps, Data (Input and Output), double click on Table.
  3. In the work area, click on the Table step. At the bottom, under Table Properties, click the file cabinet next to the Library field. Click on VST under Libraries, then select the PVA_DONORS data on the right and click OK. Note that it is also possible to copy the PVA_DONORS node from a previously created flow and paste it into the new flow.
  4. Under SAS Steps, expand Statistics, then double click on t-Tests.
  5. Next, move your mouse towards the edge of the PVA_Donors node until the pointer becomes a hand icon. Click and drag an arrow from PVA_DONORS node to the box on the left side of the t-Tests node (the input port). Now t-Tests is connected to the PVA_Donors data.
  6. Click on the t-Tests node, then select the Data tab. Verify that One-sample test is selected under the t-test field.
  7. Click the plus sign for the Analysis variable field. Assign Donation_Amt as the analysis variable. Click OK to close the Column Selection window.
  8. On the Options tab, in the Alternative hypothesis: mu ≠ field, enter 100.
  9. Under Normality Assumption, clear the option to conduct Tests for normality.
  10. To select plots, expand the Plots property and use the drop-down menu to choose the option Selected plots. Select the option to display a Confidence interval plot in addition to the default plots of histogram, a box plot, and a normality plot.
  11. Save the flow to the VST folder as ttest.flw.
  12. Run the flow and examine the results.

    13. The mean donation amount is $101.60, and the associated t statistic value is 4.44. The p-value is <0.0001. Assuming a significance level of 5% (that is, alpha=0.05), you find that the p-value is less than alpha. Thus, you reject the null hypothesis and conclude that the hypothesized population mean donation amount of $100 is significantly different from the sample mean of $101.60. Scroll further to examine the confidence interval plot. It is a horizontal line ranging from 100.9 to 102.4 to graphically represent the 95% confidence interval around the mean estimate of the donation amount.


    Image of Mean of Donation_Amt.

    Notice the vertical reference line drawn at 100 (the hypothesized value). This reference line lies outside the confidence limit, which clearly indicates that the mean value in the sample is statistically different from $100 at an alpha level of 0.05.

    Next, the histogram and the q-q plot for donation amount do not show any severe departures from normality although there are some potential outliers present in the data. Therefore, the Student's t test is valid.

    At this point, your brain might be struggling to accept whether a difference of $1.60 between the hypothesized value and the sample mean is significant when the Donation_Amt variable ranges from $0 to $211.9. Well, the interpretation that you just made from the t-test is based on the data in hand and the significance level chosen. Recall our discussion on the over-reliance on the arbitrary cutoff of a 5% significance level and the sample size. In inferential statistics the 5% level of significance might work well given the sample size. However, in a big data context, it is not always appropriate. Large sample sizes result in smaller p-values, and in that case, you tend to reject the null hypothesis almost every time. Thus, you better select a very small significance level cutoff when using the p-value approach in the big data world. In this example, we have a significantly large data set (from a statistical perspective) with 4843 observations. Thus, using a 5% level of significance is highly likely to produce some statistically significant results.

    Let's see the impact of using a smaller significance level cutoff on the inference that we draw and the confidence intervals.

    Changing the Values of the Level of Significance in the t-Test

  13. Navigate to the VST course folder and double click on the SAS program file Ttest change alpha.sas.
  14. Look at the SAS program to see the specified alpha level for this ttest:



    15. Open code or syntax in a separate window. 

    proc ttest data=VST.PVA_DONORS sides=2 h0=100 plots(only showh0)=
	(summaryPlot intervalPlot qqplot) alpha=0.000001;
   var Donation_Amt;
run;

  15. Click the + Code to Flow button and add the program to the previously saved ttest flow.


    16. Image of dding program to the previously saved ttest flow.

  16. Return to ttest.flw. Right click on the ttest change alpha node and click Run node. Save the flow and examine the Results.


    Image of Run Node flow.

    Note that neither the t statistic value nor the corresponding p-values have changed in comparison to previous run results. What changes is the confidence interval for the mean donation amount and the statistical inference.


    Image of run results.

    Image of Mean of Donation_Amt.

    Now you see that the 99.9999% confidence limit values for the mean range from 99.83 to 103.4, which clearly includes the hypothesized value within it. The same interpretation can be drawn from the confidence intervals plot. You see that the reference line at $100 is contained within the confidence limit. Thus, you would reach the conclusion that the mean donation amount is not statistically different from $100.

    Note that there is nothing special about considering a significance level of 5% always. It should depend on the problem that you are studying and the sample size.