How to track shared bandwidth usage in EC2 or Elastic IP instances?

Question

I am looking for a way to track bandwidth usage based on each instance or based on elastic IP. Amazon doesn't seem to offer these metrics. You can get overall bandwidth to / from bandwidth through your reporting mechanisms, but this includes private network bandwidth and account width. You can use cloudwatch to collect deeper metrics, but they also combine public and private bandwidth. We study our own, but your servers are built with one interface, and any elastic IPs are NATd for that interface. Since everything passes through one interface, everything is combined.

Does anyone have any suggestions? Have you ever encountered a similar problem? This is a single interface linux server environment from which you must determine the use of public bandwidth.

Answer 1

Answering an old question in favor of Googlers.

We encountered a similar problem and "solved" it using iptables counters, making us that all outgoing traffic, which is private, will be on the IP address 10.0.0.0/8, and the rest will be public traffic. You can also track input for other purposes; Of course, only outgoing public traffic is paid.

So create some counters:

iptables -A INPUT -s 0.0.0.0/0 --> Total incoming traffic iptables -A INPUT -s 10.0.0.0/8 --> private incoming traffic iptables -A OUTPUT -d 0.0.0.0/0 --> Total outgoing traffic iptables -A OUTPUT -d 10.0.0.0/8 --> private outgoing traffic 

Check counters:

  iptables -nv -L INPUT --> counters about incoming traffic iptables -nv -L OUTPUT --> counters about outgoing traffic 

NOTE. When you use the values, you get private and TOTAL: therefore, to get the publication, subtract all subtraction from Total before using it for anything.

You can also reset the counters if you do not want to report aggregate bandwidth:

  iptables --zero INPUT --> clear counter iptables --zero OUTPUT --> clear counter 

The following (ugly) bash script that will pop this information in Ganglia if you have already created counters:

  #!/bin/bash OUTPUT_PUBLIC=`sudo iptables -nvx -L OUTPUT | head -3 | tail -1 | tr -s [:blank:] |cut -d' ' -f3` OUTPUT_PRIVATE=`sudo iptables -nvx -L OUTPUT | tail -1 | tr -s [:blank:] |cut -d' ' -f3` let OUTPUT_PUBLIC=$OUTPUT_PUBLIC-$OUTPUT_PRIVATE sudo iptables --zero INPUT sudo iptables --zero OUTPUT gmetric -n "public_outbound_traffic" -v $OUTPUT_PUBLIC -t uint32 -u "bytes" gmetric -n "private_outbound_traffic" -v $OUTPUT_PRIVATE -t uint32 -u "bytes" 

Run this in cronjob, just make sure the cronjob frequency matches the message frequency of your ganglia (or else handles possible inconsistencies).

Hope this helps someone.

Answer 2

I don’t think there is an easy way to do this if Amazon doesn’t provide it, but can have a log in every machine every time we load or load something,