Where to Look First? Troubleshooting Environment issues in SAS 9 and SAS Viya

When troubleshooting various applications inside SAS software, it can be troublesome to know where to start. Much like a person getting sick, different problems can manifest themselves in different ways. Let’s explore some of the most common issues in both SAS 9 and Viya.

SAS 9.4

Let’s explore the SAS 9 Middle-Tier Environment. Once the components are installed and configured correctly, the middle tier allows users to interact with the SAS Intelligence Platform through a variety of web-based interfaces, such as SAS Studio with just a URL and proper credentials.

Behind that user-friendly web interface are several components working together, most notably:

· SAS Web Application Server

· SAS Web Server

· SAS Cache Locator

· SAS JMS Broker

· Java Runtime Environment

· SAS Environment Manager

For an explanation on what these components do, you can visit the SAS support page: SAS 9.4 Middle-Tier Software Components.

SAS is great for the vast amount of log files that it creates, but it can also be overwhelming. To keep the focus on the middle-tier, the following link from the SAS support page lists the location of logs specific to the middle-tier components: Middle-Tier Log Locations.

Let’s start at the Cache Locator as an example. The Cache Locator is used by SAS web applications to locate and connect to a distributed cache. This allows SAS web applications to be aware of other sessions and to share application data.

Sometimes you may run into obvious problems, such as the service itself simply not starting or shutting down unexpectedly. In this case, based on the documentation link above, we would know to head over to

/SAS-configuration-directory/Lev1/Web/gemfire/instances/ins_41415

directory to begin assessing the gemfire.log.

Make sure to keep an eye on these logs as they can grow to unmanageable size. This can happen if the Cache Locaters try to communicate with each other in a multi-tiered deployment. The Cache Locator service won’t be down, but if you were to look at the gemfire.log you would see the logs being filled with errors such as:

<WAN Locator Discovery Thread> tid=0x51] Locator discovery task could not exchange locator information <hostname>[41415] with <hostname>[41415] after 951 retry attempts. Retrying in 10,000 ms

What does a HTTP 500 Error Mean?

Is the error occurring with one particular application? If so, you can start at the specific application log. For example, if you see an HTTP 500 when trying to access the SAS Studio URL, head over to its log at

/SAS-configuration-directory/Lev1/Web/Logs/SASServer1_1/SASStudioMidTierx.x.log.

If not, you may need to review additional logs. Look at logs for the SAS Web Application Server, such as the server.log and gemfire.log, to find out if the server was unresponsive long enough to be removed from the Cache Locator. This could prompt us to make changes to Java memory heap sizes or modifying the Gemfire member-timeout parameter.

Note how reviewing Web Application Server logs led us back to another Mid-Tier component, the Cache Locator. Understanding how all the various components that are involved with the Mid-Tier work together is key to keeping a healthy environment. It is especially important to giving you a great starting path when diagnosing you SAS systems.

SAS Viya

Next, let’s look at a very commonly used application in SAS Viya: SAS Studio V.

SAS Studio V provides an interactive GUI for running and maintaining data sets and SAS code, similarly to SAS Studio. In addition, it also allows the user to run a CAS action around their SAS code, allowing you to receive the full benefits of cloud computing that Viya has to offer.

Things can go wrong, however. The first place to look is the StudioV standardized log:

/opt/sas/viya/config/var/logs/studiov/default/

Looking here should provide you an idea of what type of problem you are dealing with. However, it is unlikely that just this log will give you the full scope of the issue. You might see generic error messages relating to HTTP codes, or access/authentication issues, or even inaccessible locations on the operating system. Often, the symptoms show in the application even though the issue is not with StudioV at all. For example, login and authentication issues may not even be a StudioV problem at all. In these cases, you would want to first check out the SASLogon logs located here:

/opt/sas/viya/config/var/logs/saslogon/default/

Here’s an example of an error message you might see in the saslogon log:

org.springframework.web.client.HttpClientErrorException: 403

This message is a generic indicator that the client connecting to the webapp does not have the necessary permissions to access the URL.

In addition to a log message, you will often see an error displayed on the webapp screen you are logging into, indicating an error code in line with the issue. Codes 403 and 401 are the typical return codes you see relating to authentication.

As mentioned before, the StudioV log will not always give you all the information. Especially during run-time inside StudioV, there are additional logs you can investigate that might help further diagnose the issue:

/opt/sas/viya/config/var/logs/compute/default/sas-compute*.log

/opt/sas/viya/config/var/logs/launcher/default/sas-launcher*.log

/opt/sas/viya/config/var/logs/runlauncher/default/sas-runlauncher_console*.log

/opt/sas/viya/config/var/logs/tklauncher/default/tklauncher*.log

/opt/sas/viya/config/var/logs/compsrv/default/ComputeServer*.log

/opt/sas/viya/config/var/logs/compsrv/default/ComputeServer*pgm.log

Universal to SAS 9.4 and Viya

Finally, webapps always have the “web” component to them, and therefore a networking aspect. HTTP error codes can provide a hint as to what the problem is, such as a 503 or a 404.

A few of the most common HTTP codes are 503 (Service Unavailable), 404 (Not found), and 403(Forbidden). The 500 series codes generally pertain to server-side issues – such as the requested webapp service being down.

While the 400 series codes generally pertain to client-side issues – such as having the wrong URL or being denied access by the firewall. This is helpful to know, as connection protocols and HTTP codes are the same between SAS 9, SAS Viya, and virtually any other software that has networking involved.

A good way to ascertain if the issue is web related is to open your browsers developer tools and produce a HAR file. This will monitor and record all web traffic and produce a type of log that shows successful and unsuccessful transmissions. Every transmission has a code next to it; pairing this with the server-side logs can usually point you in the direction of the root cause of your problems.

Diagnosing issues in a SAS environment can be troublesome but knowing where to start looking for your problem is the first step to any resolution. The above steps touch on a couple of common issues and where to start troubleshooting.

Similar to a person, an environments’ “illness” can show symptoms in many ways and knowing these symptoms can help to diagnose whatever issues might be plaguing your system. Learning where to look to successfully diagnose issues in your system will ensure your team can continue operating at full capacity.