Amazon Kinesis Data Analytics · Finder + guided fix

Kinesis Analytics Optimize Logs

Optimize

Reduce verbose logging on Kinesis Data Analytics (Managed Service for Apache Flink) applications to cut CloudWatch Logs costs. CloudFix identifies applications that log more than a stable workload needs.

What happens, in order.

Finder · read-only
Finds the opportunity
Scans your connected accounts with a read-only role. It can see resource metadata and usage; it can’t change anything.
You
Review in Recommendations
It appears in your Recommendations view with what it would save.
Guided fix
You make the change
You apply it yourself, following the fix’s step-by-step instructions; the write-up below explains it.

Finder role

Read-only. It can see resource metadata and usage. It can’t change anything.

Fixer role

Not used here: this one is a guided fix, so the change is yours to make.

Saves by

This Finder identifies Kinesis Data Analytics (Amazon Managed Service for Apache Flink) applications whose monitoring configuration sends verbose logs to CloudWatch Logs. Lowering the log level on stable production applications reduces CloudWatch ingestion and storage costs while keeping warnings and errors visible.

Amazon Kinesis Data Analytics, now Amazon Managed Service for Apache Flink, can send application logs to Amazon CloudWatch Logs at several levels of detail. This CloudFix Finder identifies applications whose logging is more verbose than a stable production workload needs, so you can turn it down and stop paying CloudWatch to ingest and store logs nobody reads.

Overview

Problem Statement

Each Managed Service for Apache Flink application has a monitoring configuration that sets how much it logs (from ERROR up to DEBUG) and how detailed its CloudWatch metrics are (from the application level down to individual operators). Verbose settings are useful while an application is being built or debugged, but they are often left in place in production, where every log line is ingested and stored by CloudWatch Logs and billed accordingly.

Solution

CloudFix identifies Flink applications running with verbose logging and recommends a lower log level for stable workloads, keeping the warnings and errors your team relies on. Reducing the volume of log data sent to CloudWatch lowers ingestion and storage costs without changing how the application processes data.

AWS Services Affected

Amazon Managed Service for Apache Flink

Amazon CloudWatch

How It Works

The Finder reviews your Managed Service for Apache Flink applications and their CloudWatch Logs usage, read-only, to identify applications whose logging level produces far more log data than a stable workload needs. Each application appears in your Recommendations with its potential savings and a suggested logging configuration.

This is a guided fix that you apply. In the AWS Console, the AWS CLI or your infrastructure-as-code templates, update the application’s monitoring configuration to the recommended log level (for example WARN or ERROR instead of INFO or DEBUG), keeping the metrics level your dashboards and alarms depend on.

FAQ

Which applications does this apply to?

Applications running on Amazon Managed Service for Apache Flink (formerly Amazon Kinesis Data Analytics) that send logs to CloudWatch Logs at a verbose level.

Is the fix automatic after accepting the recommendation?

No. This is a guided fix: you change the application’s monitoring configuration yourself, or engage CloudFix Professional Services to do it for you.

Does this fix require downtime?

Changing an application’s configuration updates the application, and a running application restarts to apply it. With snapshots enabled it resumes from its latest state, but plan the change for a quiet period.

Can changes be rolled back?

Yes. You can raise the log level again at any time, for example while investigating an issue, by updating the monitoring configuration.

What are the potential savings?

Savings depend on how much log data the application produces today. CloudFix shows the projected savings for each application in your Recommendations.

Ready to start saving on AWS? See how much you could cut from your cloud bill with a free cost optimization assessment, or explore CloudFix automated Finder/Fixers that eliminate waste across 30+ AWS services.

Browse all 110+ finders

See whether this one applies to your account.

A free savings assessment reports which fixers can be applied to your environment. Results typically within 24 hours.

Get a free savings assessment See how CloudFix works