NetQ provides the ability to codify playbooks and extend NetQ with custom commands for use cases specific to your network.
The summary of steps required to do this is a follows:
- The extensions must be written in Python or Cython.
- The commands need to be added must use
- The .py file (or the compiled .so if using Cython) is now copied to /usr/lib/python2.7/dist-packages/netq_apps/modules/addons.
- Enable the add-ons with the
netq config add addonscommand
- Check that your command works by typing
Sample File with Custom Command
To help you get started, here is the Hello World of NetQ command extension:
Let's break down each part of the code.
Command Specification With Help
The lines at the start of the file within the triple quotes (''') constitute what is called the docstring of the file or module.
network-docopt, the Python library that builds the command parser for NetQ, uses the information provided in the docstring. Specifically, everything between Usage and Options is considered a command specification. In this case,
netq hello is the only command specified in the file. The command MUST start with the word
netq command follows the following structure:
For example, here is the sample for
The <hostname> option is used to filter results to just the specified host; hostname can also be a regular expression. The <verb> is show, the <object> is vlan and the remaining parameters are filters to viewing the data.
For example, if you wanted to extend hello world by passing an optional greeting, modify the usage to be:
network-docopt understands a few parameter types and validates them before passing them to your code. Some common ones are:
- <hostname>: A host known to NetQ
- <remote-interface>: An interface on the specified host known to NetQ
- <text>: Any free text, but has to be a single word or delimited within quotes
- <ip>, <ip/prefixlen>: IPv4 or IPv6 address, with prefix length in the second case
- <ipv4>, <ipv4/prefixlen>: IPv4 address, with prefix length in the second case
- <ipv6>, <ipv6/prefixlen>: IPv6 address, with prefix length in the second case
- <wildcard>: All the remaining text
- Valid number range: Such as <1-4096> to limit the allowed range
So in the VLAN example above, specifying a VLAN value outside the 1-4096 range results in an error, with command unknown and a help message indicating that you need to specify a value between 1 and 4096. For hosts and interfaces used with <hostname> and <remote-interface>, NetQ automatically provides tab completion.
To display meaningful help associated with a keyword, add the help for the command via the Options section. In the example code above, the object hello has the help text "Hello world experimental". This text is displayed when the user types
netq <TAB>, as shown in the following example:
Any help you provide here overrides the help provided for the keyword by a module loaded previously.
Associating the Command with the Function
After configuring the command, you need to associate or bind that command with the function to be called when a user runs the command. This is done by using decorators to functions similar to how other CLI builders or web servers work.
First, create an instance of the class
NetqModule() called app. Then associate the function to the appropriate command via the decorator
@app.route. As shown in the example above, the function
cli_hello_world() is decorated to indicate that it is the function to call for the command
hello. The function takes two parameters: cli and netq. Usage of these parameters is discussed in the next section.
Keep in mind the following when matching the command to the function:
- If a prior binding has already been assigned to a command, the newer binding will fail. By default, modules in the core NetQ code take precedence over early access modules, which take precedence over the modules defined in addons directory.
- The command string can be as small as possible. For example, the commands
netq hello jsonand
netq hellocan be handled by different functions or by the same function. The NetQ command parser does a longest match first to determine which of the competing functions is assigned to execute a command. The command parser supports up to three string matches. In other words,
show ip addressis supported, but
show ip address jsonis not. Such longer command strings bound to a function either silently fail or a shorter string version is matched.
Using the cli and netq Parameters
The function that is called to execute a command expects to received two parameters, cli and netq, in the order shown in the example above.
cli is a dictionary containing the parameters provided by the user on the command line. netq contains the timestamps provided by the user, if any. Any other object within NetQ can be ignored. The timestamps are provided to query NetQ objects around a specific time or in a time window.
The example shows how to extract the value provided by the user at the command line from cli. Since json is a keyword, getting the key json from cli lets you to determine if the user specified json at the command line or not. If the user did not specify json at the command line,
cli.get('json') returns None, whereas if the user did specify json, then
cli.get('json') returns the string "json". Thus, if the user wants to specify a parameter along with a keyword, for example, as shown in
netq show macs [vlan <1-4096>], then the value of the VLAN to search for a MAC address can be found using
cli.get('<1-4096>'), not via
The function returns either RC_SUCCESS if successful or RC_FAIL if not. The code snippet shows how to import these values from the standard NetQ libraries.
Query the NetQ Database
While the code snippet above was sufficient to illustrate the general skeleton, if you want to extend the commands, you typically will want to add meaningful functionality such as querying the database and displaying some more meaningful information. For example, consider a new command called
show ip-routes, which displays the route information available in the database, but with a different set of fields than shown via
show ip routes. The code to do so is shown below.
Much of this code is similar to the hello world example, but the new items are discussed below.
There are two additional imports, one for netq_show and the other for Route.
netq_show is the decorator that takes care of wrapping the output in a format native to NetQ. For example, it generates the JSON for you automatically, so that you don't have to write a JSON output generator just to support JSON and you don't have to worry about supporting the tabular format, displaying rotten nodes in a different color and so on. All you have to do is generate output in the form of an
yield for every entry. The
OrderedDict ensures that the columns are displayed in the order provided in the code. The column headers are generated from the dictionary key, as are the JSON keys.
By wrapping the code with the netq_show, all these display complexities are covered for you.
Route is the database object that holds all the pertinent information about a route. Its contents are defined in the
/usr/lib/python2.7/dist-packages/netq_lib/orm/redisdb/models.py file. There are other database objects defined in the file, but this example only involves the Route object.
The Function Handler
The function that satisfies the command
show myroutes is cli_show_myroutes, and because of the decorator, takes an additional input parameter, context. It's mainly used to pass things between the main NetQ command module and the specific modules, such as this one. This particular case uses the context to update the column sizes to be used in the display.
The Query Functions
The meat of the code is the query. Objects are queried using the model of <object>.query.<query function>. This particular example uses filter as the query function, as shown by the
Route.query.filter() call. The filter function produces output filtered by the parameters specified in the keyword arguments passed. For example, the hostname keyword argument restricts the results returned by the query function to only those on the specified host. The list of keys that can be specified for an object are listed under the object's definition in the aforementioned
models.py file under the function
key_fmt(). A look at that function for the Route object shows that the key fields are: hostname, prefix, route type, routing table id, ipv4/v6 route and, If the entry is originated on this node, the protocol that added this route and the VRF name qualifier. The values returned include all the key fields plus the fields shown in the
val_fmt() function for the object.
The other useful query functions are:
query.get(): which returns just the first element matching the parameters specified.
query.latest(): which returns the latest element matching the parameters specified, and does not take any time parameters.
query.count(): which returns a count of the matching elements instead of the elements themselves.
The filter query functions return an iterator and thus is lazy about retrieving data from the back end. You can stop whenever you want in the iteration.
query.latest() both return a single object of the type the query is on while
query.count() returns an integer.
Inevitably when writing code, coding errors need to be debugged and the fixes tried again. When a module doesn't load or returns an error, it is reported in the
netqd.log, usually kept under
/var/log (unless you modified the location). Deploying the module on one node doesn't mean it is automatically available on all nodes. You must copy it to all the required nodes.
To reload the modules after making fixes, run the command
netq config reload parser.
This feature is an early access feature, and must be treated as such. There may be obscure failures which will require Cumulus Networks engineering intervention to investigate. Finally, please save the modules you write. If you reinstall the
netq-apps package, your modules may get overwritten when you install the new package. One of the next releases of NetQ should provide the ability to store these modules under
/usr/local/lib, to keep them from being affected by package management.