libnetconf  0.10.0-146_trunk
NETCONF Library
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
Transaction API (transAPI)

Libnetconf transAPI is a framework designed to save developers time and let them focus on configuring and managing their device instead of fighting with the NETCONF protocol.

It allows a developer to choose parts of a configuration that can be easily configured as a single block. Based on a list of so called 'sensitive paths' the generator creates C code containing a single callback function for every 'sensitive path'. Whenever something changes in the configuration file, the appropriate callback function is called and it is supposed to reflect configuration changes in the actual device behavior.

Additionaly, transAPI provides an opportunity to implement behavior of NETCONF RPC operation defined in the data model. In case lnctool(1) finds an RPC definition inside the provided data model, it generates callbacks for it too. Whenever a server calls ncds_apply_rpc2all() with RPC message containing such defined RPC operation, libnetconf uses callback function implemented in the module.

Understanding callback parameters

Every transapi callback function has fixed set of parameters. Function header looks like this:

int callback_path_into_configuration_xml(void **data, XMLDIFF_OP op, xmlNodePtr old_node, xmlNodePtr new_node, struct nc_err **error)

void **data

This parameter was added to provide a way to share any data between callbacks. libnetconf never change (or even access) content of this parameter. Initialy content of 'data' is NULL. transapi module may use 'data' as it like but is also fully responsible for correct memory handling and freeing of no longer needed memory referenced by 'data'.

XMLDIFF_OP op

Parameter op indicates what event(s) was occured on node. All events are bitwise ored. To test if certaint event occured on node use bitwise and (&).

  • Node can be added or removed.
    • XMLDIFF_ADD = Node was added.
    • XMLDIFF_REM = Node was removed.
  • Nodes of type leaf can be changed.
    • XMLDIFF_MOD = Node content was changed.
  • Container nodes are informed about events occured on descendants. It can be distinguished whether the event was processed or not.
    • XMLDIFF_MOD = Some of node children was changed and there is not callback specified for it.
    • XMLDIFF_CHAIN = Some of node children was changed and associated callback was called.
  • Additionaly, user-ordered lists and leaf-lists are notified when change in order occurs.
    • XMLDIFF_SIBLING = Change in order. Some of siblings was added, removed or changed place.
    • XMLDIFF_REORDER = Undrelying user-ordered list has changed order.

Valid combinations of events

  • XMLDIFF_ADD and XMLDIFF_REM can never be specified simutaneously.
  • other restrictions depend on node type:
    • Leaf: exactly one of XMLDIFF_ADD, XMLDIFF_REM, XMLDIFF_MOD
    • Container: at least one of XMLDIFF_ADD, XMLDIFF_REM, XMLDIFF_MOD, XMLDIFF_CHAIN and posibly XMLDIFF_REORDER when node holds user-ordered list
    • List (system-ordered): at least one of XMLDIFF_ADD, XMLDIFF_REM, XMLDIFF_MOD, XMLDIFF_CHAIN and posibly XMLDIFF_REORDER when node holds user-ordered list
    • List (user-ordered): at least one of XMLDIFF_ADD, XMLDIFF_REM, XMLDIFF_MOD, XMLDIFF_CHAIN, XMLDIFF_SIBLING and posibly XMLDIFF_REORDER when node holds user-ordered list
    • Leaf-list (system-ordered): exactly one of XMLDIFF_ADD, XMLDIFF_REM
    • Leaf-list (user-ordered): at least one of XMLDIFF_ADD, XMLDIFF_REM, XMLDIFF_SIBLING

Ex.: Leaf processing

int callback_some_leaf(void **data, XMLDIFF_OP op, xmlNodePtr old_node, xmlNodePtr new_node, struct nc_err **error) {
if (op & XMLDIFF_MOD) {
// change configured value
} else if (op & XMLDIFF_REM) {
// leaf removed (disable service, close port, ...)
} else if (op & XMLDIFF_ADD) {
// leaf added (enable service, open port, ...)
} else {
*error = nc_err_new(NC_ERR_OP_FAILED);
nc_err_set(error, NC_ERR_PARAM_MSG, "Invalid event for leaf node /some/leaf.");
return(EXIT_FAILURE);
}
return(EXIT_SUCCESS);
}

xmlNodePtr old_node & new_node

Pointer to a particular node instance in either the old or new configuration document, in which the event was detected. When the node was removed, new node is not set and when the node was deleted, old node is not set. It is safe to traverse the whole document using these pointers, but should be used only when necessary, since transAPI itself does this for you.

Note
It is safe to traverse these nodes, but any modification will normally be lost. If you need to change some nodes, you can do so, but only in the new_node subtree (not available on XMLDIFF_REM). Then, for these changes to be written to the datastore, change 'config_modified' to 1. This variable is generated into every module.

strict nc_err **error

libnetconf's error structure. May (and should) be used to specify error when it occurs and callback returns EXIT_FAILURE. Error description is forwarded to client.

History of the transAPI versions

Each transAPI module source code is generated with the transapi_version variable set to the transAPI version supported by the code generator (lnctool(1)). libnetconf requires exactly the same transAPI version in the modules as it supports itself. However, some of the transAPI versions are kind of backward compatible, so it is possible to simply change the value of the transapi_version variable in the module source code. In that case no additional changes to the transAPI module source code are required.

Here is the list of transAPI versions with notes to the changed things and to the backward compatibility.

  • version 1
    • Initial reversion.
  • version 2
    • Allow callbacks to modify configuration data. This action is announced by the callback via the config_modified variable.
    • Changes prototype of the transAPI callbacks. It allows to return NETCONF error description structure from the callbacks.
    • Backward incompatible.
  • version 3
    • Changes prototype of the transapi_init() function. It allows the module can announce to libnetconf the initial configuration of the device when the module is loaded.
    • Changes prototype of the transAPI callbacks. The configuration data are passed to the callbacks only as the libxml2 structures. Callbacks variant passing configuration data as strings are no longer available.
    • Backward incompatible.
  • version 4
    • Callback order - the module can change the order of executing callbacks from the default 'from leafs to rootto 'from root to leafs. This is done via the callbacks_order variable. If the variable is not defined (such as in a transAPI v3 module), the default callback order is used.
    • Backward compatible.
  • version 5
    • Adds support for monitoring external files.
    • Backward compatible.
  • version 6
    • Every data callback now receives the corresponding node from both the old configuration and the new configuration. This holds for every operation except XMLDIFF_ADD (the old node is NULL) and XMLDIFF_REM (the new node is NULL).
    • Every RPC callback now receives a list of all the arguments and it is up to developers to parse them themselves. To help with this, a simple function get_rpc_node() is included in a transAPI module code.
    • Backward incompatible.

transAPI Tutorial

On this page we will show how to write a simple module for controlling an example Turing machine. The full implementation can be found in the Netopeer project.

Note
To install libnetconf follow the instructions on the Compilation and Installation page.

Preparations

In this example we will work with the data model of a Turing machine provided by the Pyang project (https://code.google.com/p/pyang/source/browse/trunk/doc/tutorial/examples/turing-machine.yang).

First, we need to identify important parts of the configuration data. Since the turing-machine data model describes only one configurable subtree, we have an easy choice. So, we can create the 'paths_file' file containing the specification of our chosen element and mapping prefixes with URIs for any used namespace.

Our file may look like this (irrespective of order):

tm=http://example.net/turing-machine
/tm:turing-machine/tm:transition-function/tm:delta

Generating code

  1. Create a new directory for the turing-machine module and move the data model and the path file into it:
    $ mkdir turing-machine && cd turing-machine/
    $ mv ../turing-machine.yang ../paths_file .
  2. Run lnctool(1) for transapi:
    $ lnctool --model ./turing-machine.yang transapi --paths ./paths_file

Besides the generated source code of our transAPI module and GNU Build System files (Makefile.in, configure.in,...), lnctool(1) also generates YIN format of the data model and validators accepted by the libnetconf's ncds_new_transapi() and ncds_set_validation() functions:

  • *.yin - YIN format of the data model
  • *.rng - RelagNG schema for syntax validation
  • *-schematron.xsl - Schematron XSL stylesheet for semantics validation

The data model can define various features and use them via the if-feature clauses. By default, all features are enabled for the validators. If you plan to to implement only a specific set (or none) of features, specify it to using the --feature` option (that can be used multiple times). The value has the following syntax:

--feature module_name:feature_to_enable

If you want to disable all features of the module, use the following syntax:

--feature module_name:

Augmenting module

When you are adding a model augmenting the original model, you have generally 2 ways of doing so:

  1. Create a new transAPI module implementing the original model with any augments, basically treating it as a single model. This way you receive a standalone transAPI module that will make the original module obsolete. lnctool(1) command:
    $ lnctool --model <original_model> --augment-model <augment_model> transapi --path <paths_for_original_and_augment_model>

  2. Create a new transAPI module implementing only the augmented parts. This way you receive an additional module that will be used together with the original module, which does not need to be modified in any way. lnctool(1) command:
    $ lnctool --model <augment_model> transapi --path <paths_for_augment_model>


However, the case when a model is augmenting an RPC in the original model is special and ONLY the first way of augmenting a module can and MUST be used.

Filling up functionality

Here we show a turing-machine simulating module. The full implementation can be found in the Netopeer project repository (transAPI/turing/turing-machine.c). The example functions and all the code is simplified and NOT thread-safe.

  1. Open 'turing-machine.c' file with your favorite editor:

    $ vim turing-machine.c

  2. Add global variables and auxiliary functions. This is completely up to you, libnetconf does not work with this anyway. For full explanation of this code look into the referenced working example. It should be called to run the Turing machine, once set up, but some parts were omitted.

    static tape_symbol *tm_head = NULL;
    static tape_symbol *tm_tape = NULL;
    static cell_index tm_tape_len = 0;
    static state_index tm_state = 0;
    static struct delta_rule *tm_delta = NULL;
    static void* tm_run(void *arg) {
    int changed = 1;
    struct delta_rule *rule = NULL;
    while(changed) {
    changed = 0;
    if (tm_head < tm_tape || (tm_head - tm_tape) >= tm_tape_len) {
    break;
    }
    for (rule = tm_delta; rule != NULL; rule = rule->next) {
    if (rule->in_state == tm_state && rule->in_symbol == tm_head[0]) {
    if (rule->out_state != 0xffff) {
    tm_state = rule->out_state;
    }
    tm_head[0] = rule->out_symbol;
    tm_head = tm_head + rule->head_move;
    changed = 1;
    usleep(100);
    break;
    }
    }
    }
    return (NULL);
    }

  3. Complete the 'transapi_init()' function with actions that will be run right after the module loads and before any other function in the module is called.

    The 'running' parameter can optionally return the current configuration state of the device as the 'transapi_init()' detects it. The configuration must correspond with the device data model and it is supposed to contain only the configuration data (defined with 'config true`). The returned data are then compared with the startup configuration and only the diverging values are set according to the startup content using the appropriate transAPI callback functions.

    We ignore it in our example - the Turing machine does not have any configuration that could be read from (depend on the state of) the system.

    Note
    After returning from 'transapi_init()' it is assumed that the current running configuration reflects the actual state of the controlled device. For instance, if the model includes some default values and the running configuration is empty, libnetconf assumes that the device is in this default state as defined in the model. If not, then you should apply the default configuration on the device in 'transapi_init()'.


    int transapi_init(xmlDocPtr *running) {
    return EXIT_SUCCESS;
    }


  4. Locate the 'transapi_close()' function and fill it with actions that will be run just before the module unloads. No other function of the transAPI module is called after the 'transapi_close()'. We free up all the temporary variables used to store the current state of the machine.

    void transapi_close(void) {
    struct delta_rule *rule;
    free(tm_tape);
    for (rule = tm_delta; rule != NULL; rule = tm_delta) {
    tm_delta = rule->next;
    free_delta_rule(rule);
    }
    }

  5. Fill 'get_state_data()' function. This function returns (only!) the state data (defined with 'config false').

    char * get_state_data(char * model, char * running, struct nc_err **err) {
    return strdup("<?xml version="1.0"?><turing-machine xmlns="http://example.net/turing-machine"> ... </turing-machine>");
    }

  6. Complete the configuration callbacks (they have the callback_ prefix). The 'op' parameter can be used to determine operation which was done with the node. Parameter 'old_node' holds a copy of node before the change and 'new_node' after the change. More detailed information about the callback parameters can be found above in the Understanding callback parameters section.

    In this code, we treat modification as a removal and an immediate addition to limit branching and simplifiy the code.
    int callback_tm_turing_machine_tm_transition_function_tm_delta(void **data, XMLDIFF_OP op, xmlNodePtr old_node, xmlNodePtr new_node, struct nc_err **error) {
    char *content = NULL;
    xmlNodePtr n1, n2;
    struct delta_rule *rule = NULL;
    if (op & XMLDIFF_MOD) {
    op = op | (XMLDIFF_REM & XMLDIFF_ADD);
    }
    if (op & XMLDIFF_REM) {
    //remove the rule from the internal list
    }
    if (op & XMLDIFF_ADD) {
    //parse the children of new_node and add the new rule to the list
    }
    return EXIT_SUCCESS;
    }

  7. Fill the RPC message callback functions with the code that will be run when an RPC message with the defined operation arrives.

    The RPC defines an optional parameter 'tape-content', so we use it if specified, otherwise use the default value.

    nc_reply *rpc_initialize(xmlNodePtr input) {
    xmlNodePtr tape_content = get_rpc_node("tape-content", input);
    struct nc_err* e = NULL;
    free(tm_tape);
    tm_state = 0;
    tm_tape = tm_head = (char*)xmlNodeGetContent(tape_content);
    if (tm_tape == NULL) {
    tm_tape = tm_head = strdup("");
    }
    tm_tape_len = strlen(tm_tape) + 1;
    pthread_mutex_unlock(&tm_run_lock);
    return nc_reply_ok();
    }

    To run the machine, we create another thread and let it run in the background. There are no parameters, so we ignore the input.

    nc_reply *rpc_run(xmlNodePtr input) {
    pthread_t tm_run_thread;
    struct nc_err *e;
    char *emsg = NULL;
    int r;
    if ((r = pthread_create(&tm_run_thread, NULL, tm_run, NULL)) != 0) {
    e = nc_err_new(NC_ERR_OP_FAILED);
    asprintf(&emsg, "Unable to start turing machine thread (%s)", strerror(r));
    nc_err_set(e, NC_ERR_PARAM_MSG, emsg);
    free(emsg);
    return nc_reply_error(e);
    }
    pthread_detach(tm_run_thread);
    return nc_reply_ok();
    }

  8. Optionally, you can set monitoring for some external configuration file.

    Let's say, that our Turing machine has a textual configuration located in the /etc/turing.conf file. libnetconf can monitor this file for modification and whenever an external application changes content of the file, the specified callback is executed. It's up to the callback function to open the file for reading and get the current configuration data.

    int example_callback(const char *filepath, xmlDocPtr *running, int* execflag) {
    // do nothing
    *running = NULL;
    *execflag = 0;
    return EXIT_SUCCESS;
    }
    struct transapi_file_callbacks file_clbks = {
    .callbacks_count = 1,
    .callbacks = {{.path = "/etc/turing.conf", .func = example_callback}}
    };

    Here is the description of the callback function parameters:
    • const char *filepath - input parameter providing the path to the changed file
    • xmlDocPtr *edit_config - output parameter to return content for the edit-config operation to change the content of the NETCONF running datastore.
    • int *exec - output parameter to set if the performed changes should cause execution of the regular transAPI callbacks. If set to 0, the changes are only reflected in the running configuration datastore, but no transAPI callback is executed.

  9. Done

Compiling module

Following sequence of commands will produce the shared library 'turing-machine.so' which may be loaded into libnetconf:

$ autoreconf --force --install
$ ./configure
$ make

Integrating to a server

In a server we use libnetconf's function ncds_new_transapi() instead of ncds_new() to create a transAPI-capable data store. Then, you do not need to process any data-writing (edit-config, copy-config, delete-config, lock, unlock), data-reading (get, get-config) or module data-model-defined RPC operations. All these operations are processed inside the ncds_apply_rpc2all() function.