This blog describes some of the opensource tools around YANG. While there exist some tools around the YANG language validation, I want to cover the bigger landscape of data modeling-driven management tools.

Data modeling-driven management is built on the idea of applying modeling languages to formally describe data sources and APIs. For some applications, this includes the ability to generate code from the models. YANG, as THE data model language of choice to create data models, is initially published in [RFC6020] as YANG 1.0, and recently updated in YANG 1.1 [RFC7950]. This new RFC does NOT obsolete YANG 1.0 [RFC6020]. What does it mean for YANG model designers: We should start using YANG 1.1 in YANG models if we need the extra YANG 1.1 capabilities. Once the YANG models are specified and implemented, a network management system (NMS) can select a particular encoding (XML or JSON) and a particular protocol (NETCONF or RESTCONF) for transport. (Yes, there are differences in capabilities between the different protocols, but that will be the topic of a future blog.) Based on the encoding and protocol selections, we can generate code, for example in Python, to access the YANG objects on the managed device, also known as the NETCONF or RESTCONF server. As we can see, the primary source of information to generate code is the set of YANG models, or YANG modules, to use the proper terminology. This is the reason why the IETF and the industry in general have been spending so much energy to specify those YANG modules as precisely as possible.

The first step in creating YANG modules is the YANG syntax validation. We have the famous pyang YANG validator, lead by Martin Bjorklund, and constantly improved by the community , especially now that the YANG 1.1 specifications [RFC 7950] are published. At the IETF, pyang is now part of the submission tool (thanks to Qin Wu and Dapeng Liu): when posting an IETF draft containing a YANG module, the YANG module language is validated. Note that no warnings or errors will prevent you from posting your draft.

In order to extract the YANG module(s) out of IETF draft for the validation, Jan Medved created the xym.py, the eXtracting Yang Module Python script for IETF drafts and RFCs. Over time, Jan improved xym.py to be more in line with rfcstrip regarding the line numbers of extracted modules.

During the IETF 97 hackathon, I added two extra validators, part of the cron jobs on my web site. Next to the pyang and confdc validators, we now have yanglint (thanks to Radek Krejčí) and the commercial yangdump-pro (thanks to Andy Bierman, from Yumaworks) and. The daily cron jobs extract all YANG modules from the IETF drafts and RFCs, with the xym tool. In some cases, it would be more appropriate to write that the script attempts to extract the YANG modules. This is a good reminder for IETF draft authors: the conventions from the “Guidelines for Authors and Reviewers of YANG Data Model Documents” RFC6087 must be followed, otherwise we can’t validate your YANG modules. The results of the validation are updated daily here. From that page, we can review the four validator outcomes for each YANG module, and also look up the draft name, email the draft authors, and download the YANG module. Downloading the YANG module is useful as the most compilers provide the YANG module line number of the specific warning or error. Indeed, it might happen that in the process of inserting the YANG module in the IETF draft (which adds headers and footers) and extracting it with xym.py, the YANG module line numbers might not match exactly. As a side note, the IETF process will need to improve to accommodate the proper treatment of YANG module and code in general in IETF drafts. Finally, this page offers the “compilation field”, which is computed based on the results of the four validators, to the best of my knowledge, as there is a little bit of analytics involved here: the possible compilation results are PASSED, PASSED WITH WARNINGS, FAILED, and NOTE.

Based on the introduction of those new YANG validators, the number of YANG modules that pass compilation recently dropped. This is fine, as the quality of the YANG modules will eventually improve, as a result. The comparison of the different validators helped tremendously in finding bugs in the validators — with many solved already in the current or future release — and helped clarifying the Guidelines for Authors and Reviewers of YANG Data Model Documents RFC 6087bis document, currently in the final phase at the IETF. On a daily basis, I contact the authors of the newly posted draft, and provide pointers to the YANG compilation errors, if any.

Note that the cron jobs do not only focus on improving the IETF YANG modules, but also cover the validation of IEEE, BBF, MEF, ONF, ODL, openconfig and sysrepo YANG modules, as you can see here.  Finally, from the same page, you can download all the IETF YANG modules.

If you want to directly test your YANG modules or IETF drafts in an online tool, before posting them, the YANG validator, developed by Carl Moberg is your friend. Right now, the YANG validation is based on pyang and confdc, but the yanglint validator will be added soon.

During one of the previous IETF hackaton, Jan Medved developed the symd.py, which generates a variety of yang module dependency graphs and output suitable for visualization with D3.js tools. A typical example, on the right, shows the importance of the ietf-routing YANG module, as it is imported by many other YANG modules. This ietf-routing YANG module is flagged as red as it’s now published as a RFC [RFC8022], as opposed to green for IETF drafts. (See the list of YANG modules issued from RFCs here). The symd.py tool was later improved by Einar Nilsen-Nygaard and then by Hari Ananthakrishnan. Hari added an extra option to contact all authors of dependent YANG models, with a message « new imported YANG model updated, please update yours». This feature is especially important for the import-by-revision YANG feature: indeed, when the imported module is updated, all dependent YANG modules must be updated.

During the IETF 97 hackathon, Joe Clarke created an interesting visual dependency tool based on the output of symd. Here is a comparison with the previous figure. In this tool, you can add any number of the YANG modules (drafts or RFCs), you can select the level of recursion, and it will generate the YANG module dependencies. While the symd output was either purely textual or static pictures, this tool has more functionalities: the ability to display the IETF draft or RFC from which the YANG module is extracted by moving the mouse over the YANG module, the ability to move the YANG modules around (for example to group the YANG modules stemming from the same document), and the ability to discover the next YANG module the community should focus on (the bottleneck is highlighted with a black circle). Finally, note that the RFCs are in red and the drafts are in green. See all options under the “graph options” pull down menu. Let’s look at this “Client/Server Configuration Model” package, which consists of those drafts, currently developed in the NETFCONF WG :

• draft-ietf-netconf-keystore-00 Keystore Model
• draft-ietf-netconf-ssh-client-server-00 SSH Client and Server Models
• draft-ietf-netconf-tls-client-server-00 TLS Client and Server Models
• draft-ietf-netconf-netconf-client-server-00 NETCONF Client and Server Models
• draft-ietf-netconf-restconf-client-server-00 RESTCONF Client and Server Models

For this package, let’s have a look at the impact analysis at https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-tls-client&modules[]=ietf-tls-server&modules[]=ietf-ssh-client&modules[]=ietf-ssh-server&modules[]=ietf-restconf-client&modules[]=ietf-restconf-server&modules[]=ietf-keystore&modules[]=ietf-netconf-client&modules[]=ietf-netconf-server&orgs[]=ietf&recurse=&rfcs=1. This URL is displayed to highlight the automation aspect of the tool.
After selecting the right option, the conclusions are in the attached picture: in order to publish this package, the community must first focus on the next bottleneck: the ietf-keystore key-store YANG model.

If the goal is to determine the maturity level of the YANG modules, it’s important to keep the RFCs in the picture. However, if the goal is to determine the next set of YANG modules to work on, and to stress the bottleneck, then the removal of the RFCs make sense (and there is a flag to remove the RFCs), as in the attached picture.

With so many YANG modules in the industry (from different SDOs – Standard Development Organizations, opensource projects, vendors, MIB generated ones, etc.), the challenge is to coordinate the development of all those YANG modules that are all specified in parallel. The first step is the ability to search all YANG modules for existing capabilities. For this purpose, Joe Clarke developed a second tool during this IETF 97 hackathon: a YANG database search. This impressive tool can search any of the YANG typedef, grouping, feature, identity, extension, RPC, container, list, leaf-list, leaf, notification for a specific string. And this string can be specified as a regular expression. As an exercise, enter “router.*id.*”, as a regular expression, searching for typedef. Sort the organization column to display the ietf organization first, and you will see three entries: two different router-id definitions, and one te-node-id. Assuming you want to know the maturity level of the relative document (if one of them is a RFC, this should be the reference, right?), you can directly select the impact analysis from the output page (see the previous tool in this blog). Browsing through the YANG module is very easy, as a direct link is also available, thanks to a new pyang plugin.

The ultimate goal in data model-driven management is the YANG Development Kit (YDK): a Software Development Kit that provides API’s that are modeled in YANG. The main goal of YDK is to reduce the learning curve of YANG data models by expressing the model semantics in an API and abstracting protocol/encoding details. YDK is composed of a core package that defines services and providers, plus one or more module bundles that are based on YANG models. Each module bundle is generated using a bundle profile and the ydk-gen tool. YDK-Py provides Python APIs for several model bundles.  Similarly, YDK-Cpp includes C++ APIs for the same bundles.  There are hundreds of YDK-Py samples and YDK-Cpp samples that you can use to get started.  You can find additional resources at the YDK portal.