Frank Habicht
For physical networking infrastructure to be established across borders and used for internet traffic, a case is still to be made, including measurement of traffic volumes. The need for supporting data was duly pointed out by Mr. Randy Bush during the above mentioned meeting. With some limitations and exceptions these measurements can be done by individual ISPs, as this is common practice in other regions of the world. However, many African networks don't have their own identifiers (ASN), most don't receive full internet routing information (full BGP view), and for most - especially the smaller - players the development and setup cost for a measurement solution is often too high.
A project of AfrISPA with the name ``Ungana'' has not yet produced results and does not seem to be too actively pursued at the moment. I had been shown some notes about it. The project described here is based only on original developments, starting 07/07/07.
At each participating IXP some networks peer openly and, it is assumed, would also peer with other networks.
The logical setup of Internet Exchange Points can be quite different from one to the other. IP addresses at the Exchange itself can be public or private, they can be globally ``seen''/routable or not. Routeservers can exist. These can be mandatory. Other servers and services can exist. Some logical components of this project can be implemented in multiple ways in order to cater for varying setups.
The project relies on local components at each IXP to collect both network reachability information and traffic flow information. These components should be provided and administered by a similar or the same person(s) as the one(s) in charge of administering the IXP. Some parts of this project require to be handled by someone who is sufficiently neutral with regards to the competing ISP peers at an Exchange. In order to produce benefit, the project should be implemented together with a number of IXPs. These will be called participating IXPs.
The project needs as input data from each IXP a list of all ip networks peering there openly. Ideally it is intended to collect this information live at a route collector receiving BGP feeds from the IXPs (similar to RouteViews project). This is preferred as changes will be effective faster and less manual intervention for updates is required. Where this is impossible, an update by email to the central project contact should be made, and an update via authenticated web submission is planned. These should contain a list of CIDR blocks. The possibility of automated retrieval from a looking glass is in consideration.
Lists of IP blocks assigned by RIRs such as AfriNIC can be obtained freely, and could be sorted by country. These are here considered inaccurate, since many networks we are aware of still use IP addresses from their upstream providers, that are registered in other countries.
Centrally, a unique number is assigned to each participating IXP.
The central server regularly compiles a list of these identifiers and the associated CIDR blocks. This list is published at a known web address for all participants to download. This information will be accessible by others, which is not posing a problem, as the data is considered to be public information. [opinions?] An alternative would be to disseminate these via BGP (turning the route collector into a route server). This would require a receiving BGP speaker on the Flow collector described later. It would also require a community or other attribute to provide information identifying the IXP at which the block mentioned is advertised.
This information will be used by the local Flow collectors at an IXP. It will also receive Netflow data from ISPs' transit interfaces - where international links terminate.The NetFlow data will be evaluated by a lookup in the netblocks tables to determine whether it involves a connection to addresses present at another IXP, and which IXP if so. For each sample interval the traffic volume (in bytes and packets) between the local ISP and any remote IXP gets added up and stored - for both incoming and outgoing traffic. Data will be stored in RRD, and made available through a protected web interface to each ISP. Also it is desirable to aggregate traffic data from all local ISPs to show aggregate traffic of this location with the other aggregation centers (IXPs).
It is intended to collect this information for several IXPs at a central server operated for the project. Preferably it would receive a BGP feed similar to how the RouteViews project of the University of Oregon operates. This information is published as a downloadable file, described below. It could also be distributed via BGP.
For this project, a BGP route collector is running at address rs.eaix.net or 41.222.63.195 . Routing tables are also published as a data file and frequently updated.
A daemon program running on that server will receive the Netflow information from the local ISPs' routers. store the information temporarily and add information showing which ISP it came from. Following that another program, that has access to the Routing tables obtained from the central route server, will evaluate the stored netflow data and classify for each if it is communication to or from a network at another IXP.
For each statistics interval of 5 minutes, the amount of traffic any ISP is exchanging with networks at each of the other IXPs is recorded, as bytes incoming and outgoing as well as packets incoming and outgoing. An aggregate for all ISPs at the specific IXP is also calculated.
Programming techniques that scale for a growing number of ISPs sending flows as well as many IXPs and prefixes to look up should be used.
Specific implementation depends on vendor and model of the equipment and software, one example is below.
ip flow-cache timeout inactive 120 ip flow-cache timeout active 5 ! interface upstream-external-link ip route-cache flow ! ip flow-export source FastEthernet0/0 ! an internal interface ip flow-export version 5 ip flow-export destination <your-existing-flow-collector> <port> ip flow-export destination <ixp-Vipimo-flow-collector> <port>
193.220.232.0/23 194.9.64.0/23 194.9.82.0/23 195.202.64.0/19
#IXP_id CIDR 1 216.6.26.0/24 ... 1 216.104.206.0/23 2 41.220.128.0/20 ... 3 217.199.153.252/30 4 41.220.0.0/20 4 41.210.128.0/18
install flowd from ports
config /etc/flowd.conf
make flowd start automatically on server startup
it will logflow info into a binary file
periodically (every 5 minutes), a script should be run that will:
- move the current log file
- send a SIGUSR1 to the flowd process ( kill -USR1 `cat /var/run/flowd.pid`)
- evaluate the binary flow log file by:
flowd-reader <logfile> ... | special program evaluating and updating RRDs
- then move the logfiles to archive or delete them
periodically (daily?) a file
196.223.0.0/16