Introduction

Amazon Virtual Private Cloud (VPC) Flow Logs is a feature in AWS that allows you to capture detailed information about the IP traffic going to and from network interfaces within your VPC. By enabling VPC Flow Logs, you can gain insights into network performance, security, and troubleshooting, as well as perform compliance monitoring and audit requirements. Flow Logs capture information about both accepted and rejected traffic, providing a granular view of how resources communicate within the VPC.

In this article, we will explore the different formats of VPC Flow Logs, their key characteristics, common use cases, and potential limitations.

Key Features and Characteristics

VPC Flow Logs generate logs that provide a record of network traffic within your VPC. Each log contains various fields that can help you monitor and analyze your network. The following are the key features and characteristics of VPC Flow Logs:

• Granularity: You can capture logs at different levels:
  • VPC Level: Logs traffic for all instances within the VPC.
  • Subnet Level: Logs traffic for all instances within a particular subnet.
  • Elastic Network Interface (ENI) Level: Logs traffic for a specific ENI.
• Flexible Storage Options: Flow logs can be stored in different AWS services, such as:
  • Amazon CloudWatch Logs: Ideal for real-time monitoring and alerts.
  • Amazon S3: Useful for long-term storage and compliance purposes.
• Customizable Filtering: You can choose to capture only the traffic that is accepted, rejected, or both.
• Flow Log Formats: VPC Flow Logs are provided in several formats, allowing you to choose the one that best suits your analysis and use case.

Flow Log Formats

AWS VPC Flow Logs offer different log formats, each with varying levels of detail. Below are the available formats:

1. Default Log Format

The default log format captures a predefined set of fields that provide basic information about the traffic, including the source and destination IP addresses, source and destination ports, protocol, and action (accepted or rejected).

Here is the structure of a default log record:

version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status

• Fields Explained:
  • version: Version of the flow log format.
  • account-id: The AWS account ID.
  • interface-id: The network interface ID.
  • srcaddr: The source IP address.
  • dstaddr: The destination IP address.
  • srcport: The source port.
  • dstport: The destination port.
  • protocol: The IANA protocol number.
  • packets: The number of packets transferred.
  • bytes: The amount of data in bytes.
  • start / end: The start and end times of the flow.
  • action: Whether the traffic was ACCEPT or REJECT.
  • log-status: Indicates the status of the flow log (e.g., OK, NODATA).

2. Extended Log Format

The extended log format captures additional fields that provide further details about the flow, such as the TCP flags and additional metadata. The extended format allows for deeper analysis, particularly for performance optimization and security incident analysis.

version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status tcp-flags instance-id az-id sub-id type

• Additional Fields:
  • tcp-flags: Provides information about the TCP flags set in the packet.
  • instance-id: The instance ID associated with the traffic.
  • az-id: The availability zone of the resource.
  • sub-id: The ID of the subnet where the flow originated or was destined.
  • type: Whether the interface is an internal or external type.