Problem

After installing the StackState AWS (legacy) StackPack, the AWS synchronization is in the Waiting for Data state and no data is coming in from AWS.

Troubleshooting

Common configuration problems

Here are a few common configuration problems when installing the AWS (legacy) StackPack:

• there are various installation scenarios for the AWS (legacy) StackPack with different security policies. Have you selected and applied the right policy?
• does the lambda function require any proxy configuration in order to access the AWS API?
• is the security group that the lambda functions are a part of allowing the lambda to connect to the AWS API? For example, is it allowed to access port 443?
• if the URL that the lambda function uses to communicate to StackState is using a hostname, is there a DNS available to resolve that hostname? Alternatively, you can specify an IP address in the StackState URL instead.

Synchronizations & Datasources

If the above items don't resolve the issue, let's try to pinpoint where the problem is exactly. We can do this by testing out a few things via the StackState GUI.

Let's see if StackState has the expected AWS configuration in the form of Synchronizations and Datasources.

In StackState, navigate to Settings > Topology Synchronisation > Synchronization and verify that there is a synchronization called AWS there.

Next, on Settings > Topology Synchronisation > Sts Sources, there should be an AWS datasource. If it is there, select Edit from the triple-dot menu and test the connection using the Test Connection button.

In Settings > Telemetry Sources > CloudWatch Sources, there should be an CloudWatchSource datasource. If it is there, select Edit from the triple-dot menu and test the connection using the Test Connection button.

Credentials

Once we have some idea of whether the StackState has the right configuration setup and which parts are working or failing, verifying the credentials is a useful next step. 

In StackState, navigate to StackState > StackPacks > AWS (legacy) StackPack. Validate that the AWS Access Key ID and AWS Secret Access Key values are correct. If you specified an AWS Role ARN, also make sure it is correct.

In AWS, navigate to each of the 3 StackState lambda functions (StackState-Topo-Cron, StackState-Topo-CWEvents  and StackState-Topo-Publisher) and verify that they have the right configuration. On the Configuration tab, scroll down to check the API_KEY and STACKSTATE_BASE_URL variables.

AWS Logs

If the configuration is correct, the next stop is the logs for the StackState functions running in AWS. There are three such functions, namely  StackState-Topo-Cron, StackState-Topo-CWEvents  and StackState-Topo-Publisher.

The log files for these functions can be reached via the lambda function. Navigate to the function and visit Monitor > View logs in CloudWatch.

Any errors appearing in the logs could be the cause of the issue.

StackState Logs

If there are no errors on the AWS side, let's check out the StackState logs.

The AWS synchronization logs are accessible here:

http://localhost:7070/api/logs/sync/sync.AWS?limit=100

You will need to replace localhost with the name of the StackState instance you are working with.

If the AWS synchronization has a non-standard name, you can find all logs here:

http://localhost:7070/api/logs

Topics

If the log files don't yield any clues, we will need to dig a bit deeper into the internals of StackState. StackState collects data from external sources in a topic on the Kafka bus. 

Using the URL:

http://localhost:7070/api/topic

we can see all Kafka topics. There should be a topic with the prefix "sts_topo_aws_".

Using the StackState CLI, we can also inspect messages on the topic to verify the instance is receiving data. See our documentation for the instructions.

Need help?

If the above doesn't resolve the issue, contact our support team for help. Please provide them with the logs mentioned above.