Monday, March 20, 2017

OpenWRT Modules: JUCI

JUCI = JavaScript for UCI (Unified Configuration Interface): the Web UI interface to the UCI (or, simply "The GUI front end" of OpenWRT).  Mainly developed by Martin K.  Schröder <mkschreder.uk@gmail.com>
Foundation: The jQuery and angularJS famework.

About
URL
JUCI GIT Repohttps://github.com/mkschreder/juci
JUCI Documentation in HTML formhttp://mkschreder.github.io/juci
overview of how juci workshttp://mkschreder.github.io/juci/manual/how-juci-works.html
AngularJS
AngularJS version 1 Documentationhttps://docs.angularjs.org/guide

Client Browser

Must be HTML5-enabled browser with Javascript enabled.

Principles

OpenWrt's central configuration is split into several files located in the /etc/config/ directory. Each file relates roughly to the part of the system it configures. 

Flow

  1. Modify config file (a file in /etc/config)
  2. restart service that uses that config file (/etc/init.d/<service script>)

Directory Structure

TOPDIR=$HOME/work/raptor/openwrt
As defined in $TOPDIR/feeds/juci/juci/Makefile, the tarballed juci package is downloaded into $TOPDIR/dl/juci-<ver#>.tar.gz
and untarred into $TOPDIR/build_dir/target-arm_cortex-a9_uClibs-<ver>

MAKE
====
Steps done in pakage/feeds//Makefile:
TARGETNAME=target-arm_cortex-a9_uClibc-0.9.33.2_eabi
PKG_BUILD_DIR=$TOPDIR/build_dir/$TARGETNAME/
mkdir -p ${PKG_BUILD_DIR}
cp $TOPDIR/package/feeds//src/* ${PKG_BUILD_DIR}/
make -C $TOPDIR/staging_dir/${TARGETNAME}/usr/src/juci/ MODULE=${PKG_BUILD_DIR}

Files

To see the latest pull of JUCI into our branch, check PKG_VERSION in $TOPDIR/feeds/juci/juci/Makefile
In $TOPDIR/target/linux/bcm5301x/base-files/etc/uci-defaults/config:
  • content-filter
  • juci
  • srg-diagnostics
  • system
  • user-interfaces

In $TOPDIR//packages/juci-full-/files/etc/config/:
  • capo

In $TOPDIR//packages/juci--family-shield/files/etc/config/family-shield/:
  • family-shield

In $TOPDIR/package/base-files/files/etc/config/:
  • iptv
  • network
  • ntra
  • webdav
  • wifi-insight

In $TOPDIR/network/services/ipset-dns/files/:
  • ipset-dns.config (installed as ipset-dns in target's /etc/config)


To do clean build of the whole JUCI + Juci modules:

Building Juci
make package/feeds/juci/juci/{clean,prepare,compile} package/index V=s

JUCI html files are compiled into the *.js files by command:

HTML to JS
./scripts/juci-build-tpl-cache


(The Makefile above converts html files into *.js and put in tarball file named: node_modules.tar.gz

JUCI in feed

This subdirectory is used when we want to create a JUCI patch.  We git pull the juci module into $TOPDIR/feeds/juci/juci. 

See Creating patches to JUCI core modules

The following script might be useful to do the steps as described above

JUCI Checkout
#!/bin/bash
if [ $# -lt 1 ]
then
    echo ""
    echo "$0 <juci version/tag>"
    echo ""
    echo "Current version for SOS: v2.16.09"
    exit
fi
tag=$1
cd ~/work/raptor/openwrt
./scripts/feeds update juci
pushd feeds/juci
git submodule init
git submodule update
cd juci/src
git checkout ${tag}
popd
make package/juci/clean V=s && make V=s


Once we've done the script above, we can start editing files needed.  Once we're done, just create a patch (assuming we're already in directory $TOPDIR/feeds/juci/juci/src):

Creating patch
git diff -p --stat > ../patches/01x-my-patch-name.patch


JUCI Components

  • JUCI core (juci.js)
  • Plugins (for example: js/50-juci-ddns.js, etc.)
  • Theme *.js

The plugins are located (to be exact: extracted to) $BUILD_DIR/$TARGETNAME/juci-<juci-ver#>/plugins/
For example, for menu "WIFI", the plugin is in $HOME/work/raptor/openwrt/build_dir/target-arm_cortex-a9_uClibc-0.9.33.2_eabi/juci-fcfb1fcd85735c69ffad89a9d3d9b6c26d3df810/plugins/juci-openwrt-wireless

JUCI on Target Device

The JUCI config file is /etc/config/juci, which contains menus and submenus
To tell juci to reload, type juci-update

Creating JUCI Patch

Creating patches to JUCI core modules

LINKS

Juci Coding Standard
JUCI Build Environment Setup
Testing juci core patches
JUCI Debugging Tips
Creating patches to JUCI core modules
JUCI Coding Standard

OpenWRT Modules: UCI

UCI stands for Unified Configuration Interface.

UCI is a subsystem/module intended to centralize the configuration of OpenWrt.  It manages configuration data for CPE, where the configuration information is stored in form of files in /etc/config folder on CPE.

UCI Usage

To call UCI, we can either use "uci" user-space application helper or we can directly link to UCI library to call as C routine calls.

UCI Via Shell

Get the Primary SSID:
UCI Get Value
# uci get wireless.@wifi-iface[-1].ssid


Open a port in firewall for TR69 client:
UCI Create Object and Set Values
# uci add firewall rule
# uci set firewall.@rule[-1].name='Allow TR-069 from WAN'
# uci set firewall.@rule[-1].src=wan
# uci set firewall.@rule[-1].target=ACCEPT
# uci set firewall.@rule[-1].proto=tcp
# uci set firewall.@rule[-1].dest_port=7547
# uci commit firewall

Change the httpd bind port to 8080:
# uci set uhttpd.main.listen_http=8080
# uci commit uhttpd
# /etc/init.d/uhttpd restart


A nice example from carrierwrt:
#!/bin/sh
  
# Fixup easycwmp WAN interface
if "$(uci get network.wan.type 2> /dev/null)" == "bridge" ]; then
    WANIF=br-wan
else
    WANIF=$(uci get network.wan.ifname 2> /dev/null)
fi
  
uci set easycwmp.@local[0].interface=${WANIF:-wan}
uci commit easycwmp
exit 0

UCI Call via Library API

Suppose we want to have UCI like below:
UCI show easywmp
linux: config # uci show easycwmp.stun_server
easycwmp.stun_server=stun_server
easycwmp.stun_server.min_keep_alive='60'
easycwmp.stun_server.max_keep_alive='120'
easycwmp.stun_server.server_addr='stun-server.organization.org'
easycwmp.stun_server.server_port='3478'
easycwmp.stun_server.enable='1'
easycwmp.stun_server.username='00236a:00236ac231f0@qa2'

This requires us to create a config file name "easycwmp" and name the config section "stun_server":

config file
config stun_server 'stun_server'
    option min_keep_alive '60'
    option max_keep_alive '120'
    option server_addr 'stun-server.organization.org'
    option server_port '3478'
    option enable '1'
    option username '00236a:00236ac231f0@qa2'


The skeletal C code would be like below:



Skeleton
void stun_uci_read(const char *name, char *buffer)
{
    char path[50]="easycwmp.stun_server.";
    struct  uci_ptr ptr;
    struct  uci_context *c = uci_alloc_context();
    int UCI_LOOKUP_COMPLETE = (1 << 1);
    if (!c)
        return;
    if(!strcmp(name, "username"))
    {
        strcat(path, "username");
    }
    else if(!strcmp(name, "passwd"))
    {
        strcat(path, "passwd");
    }
    else if(!strcmp(name, "keepAlive"))
    {
        strcat(path, "min_keep_alive");
    }
    if ((uci_lookup_ptr(c, &ptr, path, true) != UCI_OK) ||
         (ptr.o==NULL || ptr.o->v.string==NULL))
    {
            uci_free_context(c);
            return;
    }
    if(ptr.flags & UCI_LOOKUP_COMPLETE)
            strcpy(buffer, ptr.o->v.string);
    uci_free_context(c);
}


UCI Walk Through


To walk through each option entry under the config section:


Walk Through Options
void uci_show_value(struct uci_option *o)
{
    struct uci_element *e;
    bool sep = false;
     
    switch(o->type) {
        case UCI_TYPE_STRING:
            printf("%s\n", o->v.string);
            break;
        case UCI_TYPE_LIST:
            uci_foreach_element(&o->v.list, e)
            {
                printf("%s%s", (sep ? delimiter : ""), e->name);
                sep = true;
            }
            printf("\n");
            break;
        default:
            printf("<unknown>\n"); break;
    }
}


As the rule, if we want UCI to return data in the format of [<filename>.<config name>.<type>], The declaration in /etc/config/<filename> is:
config <config-name> 'named section'
   option <option-name> <value>

'named section' can be blank (which would make the config section unnamed)

Another example:
Config name with type
# uci show wireless.r2g
wireless.r2g=wifi-device
wireless.r2g.type='broadcom'
wireless.r2g.ifname='wifi2_4'
wireless.r2g.macaddr='00:23:6a:c2:31:f4'
wireless.r2g.txpower='24'
wireless.r2g.hwmode='11g'
wireless.r2g.htmode='HT20'
wireless.r2g.country='US'
wireless.r2g.frameburst='1'
wireless.r2g.maxassoc='32'
wireless.r2g.autochannel='1'
wireless.r2g.channel='11'

Requires config file /etc/config/wireless with entry like lines below:
Alias
config wifi-device 'r2g'
    option type 'broadcom'
    option ifname 'wifi2_4'
    option macaddr '00:23:6a:c2:31:f4'
    option txpower '24'
    option hwmode '11g'
    option htmode 'HT20'
    option country 'US'
    option frameburst '1'
    option maxassoc '32'
    option autochannel '1'
    option channel '11'

UCI API

Function Name
Purpose
struct uci_context *uci_alloc_context(void)Allocate a new UCI context
void uci_free_context(struct uci_context *ctx)Free the allocated UCI context, including all of its data
void uci_perror(struct uci_context *ctx, const char *str)
Print the last uci error that occured
@ctx: uci context
@str: string to print before the error message
void uci_get_errorstr(struct uci_context *ctx, char **dest, const char *str)
Get an error string for the last uci error
@ctx: uci context
@dest: target pointer for the string
@str: prefix for the error message
Note: string must be freed by the caller
int uci_import(struct uci_context *ctx, FILE *stream, const char *name, struct uci_package **package, bool single)
Import uci config data from a stream
  • @ctx: uci context
  • @stream: file stream to import from
  • @name: (optional) assume the config has the given name
  • @package: (optional) store the last parsed config package in this variable
  • @single: ignore the 'package' keyword and parse everything into a single package
The name parameter is for config files that don't explicitly use the 'package <...>' keyword
if 'package' points to a non-null struct pointer, enable delta tracking and merge
int uci_export(struct uci_context *ctx, FILE *stream, struct uci_package *package, bool header)
Export one or all uci config packages
  • @ctx: uci context
  • @stream: output stream
  • @package: (optional) uci config package to export
  • @header: include the package header

int uci_load(struct uci_context *ctx, const char *name, struct uci_package **package)
Parse an uci config file and store it in the uci context
  • @ctx: uci context
  • @name: name of the config file (relative to the config directory)
  • @package: store the loaded config package in this variable

int uci_unload(struct uci_context *ctx, struct uci_package *p)
Unload a config file from the uci context
  • @ctx: uci context
  • @package: pointer to the uci_package struct
int uci_lookup_ptr(struct uci_context *ctx, struct uci_ptr *ptr, char *str, bool extended)
Split an uci tuple string and look up an element tree
  • @ctx: uci context (IN)
  • @ptr: lookup result struct (OUT)
  • @str: uci tuple string to look up (IN)
  • @extended: allow extended syntax lookup (IN)

if extended is set to true, uci_lookup_ptr supports the following extended syntax:

Examples:
network.@interface[0].ifname ('ifname' option of the first interface section)
network.@interface[-1] (last interface section)
Note: uci_lookup_ptr will automatically load a config package if necessary
@str must not be constant, as it will be modified and used for the strings inside @ptr, thus it must also be available as long as @ptr is in use.

int uci_add_section(struct uci_context *ctx, struct uci_package *p, const char *type, struct uci_section **res)
Add an unnamed section
  • @ctx: uci context
  • @p: package to add the section to
  • @type: section type
  • @res: pointer to store a reference to the new section in

int uci_set(struct uci_context *ctx, struct uci_ptr *ptr)
Set an element's value; create the element if necessary
  • @ctx: uci context
  • @ptr: uci pointer

The updated/created element is stored in ptr->last
int uci_add_list(struct uci_context *ctx, struct uci_ptr *ptr)
Append a string to an element list
  • @ctx: uci context
  • @ptr: uci pointer (with value)

Note: if the given option already contains a string value, it will be converted to an 1-element-list before appending the next element
int uci_reorder_section(struct uci_context *ctx, struct uci_section *s, int pos)
Reposition a section
  • @ctx: uci context
  • @s: uci section to reposition
  • @pos: new position in the section list
int uci_rename(struct uci_context *ctx, struct uci_ptr *ptr)
Rename an element
  • @ctx: uci context
  • @ptr: uci pointer (with value)
int uci_delete(struct uci_context *ctx, struct uci_ptr *ptr)
Delete a section or option
  • @ctx: uci context
  • @ptr: uci pointer
int uci_save(struct uci_context *ctx, struct uci_package *p)
save change delta for a package
  • @ctx: uci context
  • @p: uci_package struct
int uci_commit(struct uci_context *ctx, struct uci_package **p, bool overwrite)
commit changes to a package
  • @ctx: uci context
  • @p: uci_package struct pointer
  • @overwrite: overwrite existing config data and flush delta

Committing may reload the whole uci_package data, the supplied pointer is updated accordingly
int uci_list_configs(struct uci_context *ctx, char ***list)
List available uci config files
@ctx: uci context

Caller is responsible for freeing the allocated memory behind list
int uci_set_savedir(struct uci_context *ctx, const char *dir);
override the default delta save directory
  • @ctx: uci context
  • @dir: directory name
int uci_add_delta_path(struct uci_context *ctx, const char *dir)
add a directory to the search path for change delta files
  • @ctx: uci context
  • @dir: directory name
This function allows you to add directories, which contain 'overlays' for the active config, that will never be committed.
int uci_revert(struct uci_context *ctx, struct uci_ptr *ptr)
revert all changes to a config item
  • @ctx: uci context
  • @ptr: uci pointer
int uci_parse_argument(struct uci_context *ctx, FILE *stream, char **str, char **result)

parse a shell-style argument, with an arbitrary quoting style
  • @ctx: uci context
  • @stream: input stream
  • @str: pointer to the current line (use NULL for parsing the next line)
  • @result: pointer for the result
int uci_set_backend(struct uci_context *ctx, const char *name)
change the default backend
  • @ctx: uci context
  • @name: name of the backend

The default backend is "file", which uses /etc/config for config storage
bool uci_validate_text(const char *str)
validate a value string for uci options
@str: value

this function checks whether a given string is acceptable as value for uci options
int uci_add_hook(struct uci_context *ctx, const struct uci_hook_ops *ops)
add a uci hook
  • @ctx: uci context
  • @ops: uci hook ops

NB: allocated and freed by the caller
int uci_remove_hook(struct uci_context *ctx, const struct uci_hook_ops *ops)
remove a uci hook
  • @ctx: uci context
  • @ops: uci hook ops

int uci_load_plugin(struct uci_context *ctx, const char *filename)
load an uci plugin
  • @ctx: uci context
  • @filename: path to the uci plugin

NB: plugin will be unloaded automatically when the context is freed
int uci_load_plugins(struct uci_context *ctx, const char *pattern)
load all uci plugins from a directory
  • @ctx: uci context
  • @pattern: pattern of uci plugin files (optional)

if pattern is NULL, then uci_load_plugins will call uci_load_plugin for uci_*.so in <prefix>/lib/
int uci_parse_ptr(struct uci_context *ctx, struct uci_ptr *ptr, char *str)
parse a uci string into a uci_ptr
  • @ctx: uci context
  • @ptr: target data structure
  • @str: string to parse

str is modified by this function
int uci_lookup_next(struct uci_context *ctx, struct uci_element **e, struct uci_list *list, const char *name)
lookup a child element
  • @ctx: uci context
  • @e: target element pointer
  • @list: list of elements
  • @name: name of the child element

if parent is NULL, the function looks up the package with the given name
void uci_parse_section(struct uci_section *s, const struct uci_parse_option *opts,
int n_opts, struct uci_option **tb)
look up a set of options
  • @s: uci section
  • @opts: list of options to look up
  • @n_opts: number of options to look up
  • @tb: array of pointers to found options
uint32_t uci_hash_options(struct uci_option **tb, int n_opts)
build a hash over a list of options
  • @tb: list of option pointers
  • @n_opts: number of options

Examples


module initialization and deinitialization
struct uci_context myconfig_ctx;
struct uci_package myconfig_pkg;
struct uci_package *myconfig_init_package(const char *config_fname)
{
    myconfig_pkg = calloc(1, sizeof(struct myconfig_pkg));
    if (NULL == myconfig_ctx)
        myconfig_ctx = uci_alloc_context();
    if (myconfig_pkg)
    {
        // unload first before loading
        uci_unload(myconfig_ctx, myconfig_pkg);
        myconfig_pkg = NULL;
    }
    if (uci_load(myconfig_ctx, config_fname, &myconfig_pkg))
    {
        uci_free_context(myconfig_ctx);
        myconfig_ctx = NULL;
        return NULL;
    }
    return myconfig_pkg;
}
void myconfig_free_package(void)
{
    if (myconfig_ctx)
    {
        if (myconfig_pkg)
        {
            uci_unload(myconfig_ctx, myconfig_pkg);
            myconfig_pkg = NULL;
        }
        uci_free_context(myconfig_ctx);
        myconfig_ctx = NULL;
    }
}
...
myconfig_init_package("easycwmp");

TO get and set value:


Get/Set
typedef enum {
    GET,
    SET
} cmd_t;
int myconfig_get_set(const char *name, cmd_t mode, void *val)
{
    struct uci_ptr ptr;
    if ( (myconfig_ctx == NULL) || (NULL == name) || (NULL == val) )
        return -1;
    if ((uci_lookup_ptr(myconfig_ctx, &ptr, true) != UCI_OK) || (ptr.o == NULL)  || (ptr.o->v.string == NULL) )
    {
        return -1;
    }
    if (mode == GET)
    {
        if (ptr.flags & UCI_LOOKUP_COMPLETE)
            strcpy(val, ptr.o->v.string);
        return 0;
    }
    else if (mode == SET)
    {
        ptr.value = (char *)val;
        if ((uci_set(myconfig_ctx, &ptr) != UCI_OK) || (ptr.o==NULL || ptr.o->v.string==NULL))
            return -1;
        uci_commit(myconfig_ctx, &ptr.p, false);
    }
    return 0;
}


Walk-through each entry under a subtree/sub-path (e.g: under "easycwmp.stun_server").  This is basically to load a package and initialize our buffer with values retrieved from the options in the package.

Walk-through each item and do GET
struct uci_section *s;
struct uci_element *elem;
int myconfig_load_package(void)
{
    uci_foreach_element(&myconfig_pkg->sections, elem)
    {
        s = uci_to_section(elem);
        if (strcasecmp(s->type, "stun_server") == 0)
        {
            uci_foreach_element(&s->options, elem)
            {
                if (strcmp(uci_top_option(elem->e.name, "min_keep_alive") == 0)
                    myconfig->min_keep_alive = (uint32_t)strtol(uci_top_option(elem)->v.string, NULL, 10);
                else
                if (strcmp(uci_top_option(elem->e.name, "max_keep_alive") == 0)
                    myconfig->max_keep_alive = (uint32_t)strtol(uci_top_option(elem)->v.string, NULL, 10);
                else
                if (strcmp(uci_top_option(elem)->e.name, "enable") == 0)
                    myconfig->enable = (atoi(uci_top_option(elem)->v.string)) ? true false;
                else
                if (strcmp(uci_top_option(elem)->e.name, "server_addr") == 0)
                    strncpy(myconfig->server_addr, uci_to_option(elem)->v.string, sizeof(myconfig->server_addr));
                else
                if (strcmp(uci_top_option(elem)->e.name, "server_port") == 0)
                    myconfig->server_port = (uint32_t)strtol(uci_to_option(elem)->v.string, NULL, 10);
                else
                if (strcmp(uci_top_option(elem)->e.name, "username") == 0)
                    strncpy(myconfig->username, uci_to_option(elem)->v.string, sizeof(myconfig->username));
            // uci_foreach_element(...)
            return 0;
        // strncasecmp(..)
        else
            return -1;
    // uci_foreach...
}
...
myconfig_init_package("easycwmp");
myconfig_load_package();


References




(...to be continued....)