This summer will mark the IPFIX 25 years anniversary, the famous IP Flow Information eXport protocol.

The first flow-related BoF (Birds of a Feather) took place in London in summer 2001, for the IETF meeting 51. I remember that time pretty well, as it was my very first IETF meeting: so mixed feelings of “I am terrified” and “I am proud”. Since I loved (and I still do) NetFlow and knew it in details, to the point of presenting “Advanced NetFlow” at Ciscolive, I was in charge to present NetFlow version 9 in that BoF. A few months later, the IP Flow Information Export (IPFIX) working group (WG) was created, with the following goal in its charter: “This group will select a protocol by which IP flow information can be transferred in a timely fashion from an exporter to a collection station or stations and define an architecture which employs it.”

While the WG initially classified the requirements for the selection of the base protocol in the “Requirements for IPFIX”, RFC3917, a couple of us at Cisco, Ganesh Sadasivan, Vamsi Valluri, Martin Djernaes, and myself, documented the Cisco proprietary NetFlow version 9 protocol in the informational (*) RFC 3954, for the industry benefits. Once NetFlow version 9 was selected as the base protocol for IPFIX, this RFC was used as a reference. For the little story, the IPFIX version field is not specified as IPFIX “1” but “10”

The IPFIX protocol (RFC 5101) and the IPFIX information model (RFC 5102) were finally published in January 2008 as Proposed Standards. In the end, IPFIX is an improved NetFlow v9 protocol with extra features and requirements such as transport, string variable length encoding, security, or template withdrawal message.

The rest is history. The IPFIX WG successfully concluded in 2015, with the following results:
– the IPFIX protocol and information model, respectively RFC 7011 and RFC 7012, published as Internet Standards (progressed from the RFC 5101 and RFC 5102 Proposed Standards, respectively)
– a series of RFCs regarding IPFIX mediation functions (from a subsequent charter version)
– almost 30 IPFIX RFCs in total (architecture, protocol extensions, implementation guidelines, applicability, MIB modules, the first YANG module outside of NETMOD, etc…)
– the same IPFIX community worked on PSAMP (Packet SAMPling), a WG created just one year after the IPFIX, to focus on “a standard set of capabilities for network elements to sample subsets of packets
by statistical and other methods“. The PSAMP WG logically selected IPFIX to export packet sampling information, and produced four RFCs, to cover the equivalent functionality as sampled NetFlow.

Looking back in the mirror, this was a collective effort from many committed individuals. At the risk of forgetting some, let me name a few: Jürgen Quittek, Simon Leinen, Brian Trammell, Paul Aitken, Stewart Bryant, Atsushi Kobayashi, Tanja Zseby, Andrew Johnson, Elisa Boschi, Nevil Brownlee, Nick Duffield, Thomas Dietz, Gerhard Muenz. This has been a fun ride!

Is IPFIX a success?

From a specification point of view, absolutely. Granted, should we start from scratch today, we would change a few things, reflecting on years of experience. What comes to mind is a flexible IPFIX header, with potentially extra metadata to facilitate the integration of the different monitoring protocols.
From an implementation point of view, absolutely. Many vendors implement IPFIX these days, as this is THE networking data plane observability mechanism of choice.

For some lessons learned from this long IPFIX standardization, read this blog post “From NetFlow to IPFIX via PSAMP: 13 years of Standardization Explained“, where I share my views about the standardization length, the charter intent, and the mandated congestion-aware transport protocol.

What about IPFIX these days?

At the IETF 122 in Bangkok, last month, there were multiple IPFIX discussions. In fact, in the last couple of years, I have seen a renewed interest for IPFIX.

1. Requirement to Monitor New Technologies

New IPFIX Information Elements are constantly specified as the monitoring systems must monitor the newly developed technologies (Segment Routing) and/or adjacent technologies (GTP), to name a few:

• Export of MPLS Segment Routing Label Type Information in IP Flow Information Export (IPFIX), RFC 9160
• Export of Segment Routing over IPv6 Information in IP Flow Information Export (IPFIX), RFC 9487
• New IPFIX Information Elements for TCP Options and IPv6 Extension Headers, RFC 9740
• Export of GTP-U Information in IP Flow Information Export (IPFIX) (still a draft)

To be complete, the full list of IPFIX Information Elements, which counts 529 instances today, is available at IPFIX IANA registry.

What makes me happy: 3 out of the 4 specifications above have actual running code, starting very early in the process of writing the IETF draft. Already at the IETF 115, a couple of us tested, with Thomas Graf in the captain seat, what would become the future IPFIX & SRv6 RFC 9487: with running code, a live network, and a traffic engineering use case in mind. See the glory details at IETF Hackathon: SRv6 IPFIX Flow Monitoring.

In the end, there is some hope for the IETF motto: “We reject kings, president and voting, we believe in: rough consensus and running code“.

2. IPFIX IANA Registry Clean Up

With Med Boucadair, we recently took over the task to clean up the IPFIX IANA registry. Specifically, we provided updates to fix shortcomings in the description of some Information Elements (IEs), to ensure a consistent structure when citing an existing IANA registry, and to fix broken pointers, orphaned section references, etc. The updates are also meant to bring some consistency among the entries of the registry. Finally, we revised the tcpControlBits Information Element as originally defined in RFC 5102 to reflect changes to the TCP Flags header field since RFC 793

Practically, we produced two RFCs to include those improvements:

• Simple Fixes to the IP Flow Information Export (IPFIX) Entities IANA Registry, RFC 9710
• An Update to the tcpControlBits IP Flow Information Export (IPFIX) Information Element, RFC 9565, which obsoletes the previous definitions (RFC 7125)

3. Including Performance Metrics Directly in IPFIX Flow Records

Typically, the IPFIX Metering Process (find some background in “architecture for IP Flow Information Export” in RFC 5470) generates Flow Records for packet headers and characteristics observed at the Observation Point, and packet treatment at the Observation Point (for example, the QoS or the selected output interfaces). The Metering Process might also aggregate Flow Records according to RFC 7015: for example, even if it observes IP addresses, it might aggregate new Flow Record based on network prefixes (which means btw that it must access the Forwarding Information Base/FIB).

Now, we enter a new era for IPFIX, with the addition of performance metrics directly in the (aggregated) Flow Records. See this draft Export of Delay Performance Metrics in IP Flow Information eXport, currently in last call, which exports the On-Path Telemetry measured delay on the OAM transit and decapsulating nodes. The use case is easy to understand: in order to understand which customer traffic is affected, delay-related data needs to be reported in the context of the customer data-plane. This time, this is done directly in the router, exporter in the Flow Records … as opposed to (attempt to, because it’s never deterministic in case of ECMP) map synthetic traffic (TWAMP or IP SLA) to the customer data plane.

I’ll blog more about that soon: to me, this is THE right way to go forward.

From an IETF point of view, this “next-gen” IPFIX implies that we have to deal with two IANA registries, which must remain in synch: the IPFIX one and the Performance Metric one.

This vision of combining IPFIX and performance metrics was already foreseen when we created this Performance Metric with Al Morton (R.I.P.) under RFC 8911, with the Initial Performance Metrics Registry Entries RFC 8912. It just took a little longer than expected.

4. Deployments in Operators

At this specific IETF meeting, I have discussed with two large operators who wanted to deploy IPFIX at scale, for traffic monitoring. Next to some typical IPFIX security deployments, there is a clear understanding today that we must combine the different plane information to solve the closed loop vision, in a data mesh architecture:

• Data plane network visibility: where is traffic is actually going? IPFIX
• Control plane visibility: where the traffic is supposed to go? Typically BGP Monitoring Protocol (BMP).
• Management plane visibility: interface/QoS/environmental/you-name-too/… counters and some topology information, pushed with model-driven telemetry (YANG push).

In conclusion, IPFIX is against in the spotlight with new developments & renewed interest.

(*) Informational definition from RFC 2026: An “Informational” specification is published for the general information of the Internet community, and does not represent an Internet community consensus or recommendation. The Informational designation is intended to provide for the timely publication of a very broad range of responsible informational documents from many sources, subject only to editorial considerations and to verification that there has been adequate coordination with the standards process