Programming model

Overview

Programming with Duktape is quite straightforward:

Let's look at all the steps and their related concepts in more detail.

Heap and context

A Duktape heap is a single region for garbage collection. A heap is used to allocate storage for strings, Ecmascript objects, and other variable size, garbage collected data. Objects in the heap have an internal heap header which provides the necessary information for reference counting, mark-and-sweep garbage collection, object finalization, etc. Heap objects can reference each other, creating a reachability graph from a garbage collection perspective. For instance, the properties of an Ecmascript object reference both the keys and values of the object's property set. You can have multiple heaps, but objects in different heaps cannot reference each other directly; you need to use serialization to pass values between heaps.

A Duktape context is an Ecmascript "thread of execution" which lives in a certain Duktape heap. A context is represented by a duk_context * in the Duktape API, and is associated with an internal Duktape coroutine (a form of a co-operative thread). Each context is also associated with an environment consisting of global objects; contexts may share the same global environment but can also have different environments. The context handle is given to almost every Duktape API call, and allows the caller to interact with the value stack of the Duktape coroutine: values can be inserted and queries, functions can be called, and so on.

Each coroutine has a call stack which controls execution, keeping track of function calls, native or Ecmascript, within the Ecmascript engine. Each coroutine also has a value stack which stores all the Ecmascript values of the coroutine's active call stack. The value stack always has an active stack frame for the most recent function call (when no function calls have been made, the active stack frame is the value stack as is). The Duktape API calls operate almost exclusively in the currently active stack frame. A coroutine also has an internal catch stack which is used to track error catching sites established using e.g. try-catch-finally blocks. This is not visible to the caller in any way at the moment.

Multiple contexts can share the same Duktape heap. In more concrete terms this means that multiple contexts can share the same garbage collection state, and can exchange object references safely. Contexts in different heaps cannot exchange direct object references; all values must be serialized in one way or another.

Almost every API call provided by the Duktape API takes a context pointer as its first argument: no global variables or states are used, and there are no restrictions on running multiple, independent Duktape heaps and contexts at the same time. There are multi-threading restrictions, however: only one native thread can execute any code within a single heap at any time, see Threading.

To create a Duktape heap and an initial context inside the heap, you can simply use:

duk_context *ctx = duk_create_heap_default();
if (!ctx) { exit(1); }

If you wish to provide your own memory allocation functions and a fatal error handler function (recommended), use:

duk_context *ctx = duk_create_heap(my_alloc,
                                   my_realloc,
                                   my_free,
                                   my_udata,
                                   my_fatal);
if (!ctx) { exit(1); }

To create a new context inside the same heap, with the context sharing the same global objects:

duk_context *new_ctx;

(void) duk_push_thread(ctx);
new_ctx = duk_get_context(ctx, -1 /*index*/);

To create a new context inside the same heap, but with a fresh set of global object:

duk_context *new_ctx;

(void) duk_push_thread_new_globalenv(ctx);
new_ctx = duk_get_context(ctx, -1 /*index*/);

Contexts are automatically garbage collected when they become unreachable. This also means that if your C code holds a duk_context *, the corresponding Duktape coroutine MUST be reachable from a garbage collection point of view.

A heap must be destroyed explicitly when the caller is done with it:

duk_destroy_heap(ctx);

This frees all heap objects allocated, and invalidates any pointers to such objects. In particular, if the calling program holds string pointers to values which resided on the value stack of a context associated with the heap, such pointers are invalidated and must never be dereferenced after the heap destruction call returns.

Call stack and catch stack (of a context)

The call stack of a context is not directly visible to the caller. It keeps track of the chain of function calls, either C or Ecmascript, currently executing in a context. The main purpose of this book-keeping is to facilitate the passing of arguments and results between function callers and callees, and to keep track of how the value stack is divided between function calls. The call stack also allows Duktape to construct a traceback for errors.

Closely related to the call stack, Duktape also maintains a catch stack for keeping track of current error catching sites established using e.g. try-catch-finally. The catch stack is even less visible to the caller than the call stack.

Because Duktape supports tail calls, the call stack does not always accurately represent the true call chain: tail calls will be "squashed" together in the call stack.

Don't confuse with the C stack.

Value stack (of a context) and value stack index

The value stack of a context is an array of tagged type values related to the current execution state of a coroutine. The tagged types used are: undefined, null, boolean, number, string, object, buffer, pointer, and lightfunc. For a detailed discussion of the available tagged types, see Types.

The value stack is divided between the currently active function calls (activations) on the coroutine's call stack. At any time, there is an active stack frame which provides an origin for indexing elements on the stack. More concretely, at any time there is a bottom which is referred to with the index zero in the Duktape API. There is also a conceptual top which identifies the stack element right above the highest currently used element. The following diagram illustrates this:

 Value stack
 of 15 entries
 (absolute indices)

.----.
| 15 |
| 14 |
| 13 |
| 12 |      Active stack frame (indices
| 11 |      relative to stack bottom)
| 10 |
|  9 |      .---.
|  8 |      | 5 |   API index 0 is bottom (at value stack index 3).
|  7 |      | 4 |
|  6 |      | 3 |   API index 5 is highest used (at value stack index 8).
|  5 |      | 2 |
|  4 |      | 1 |   Stack top is 6 (relative to stack bottom).
|  3 | <--- | 0 |
|  2 |      `---'
|  1 |
|  0 |
`----'

There is no direct way to refer to elements in the internal value stack: Duktape API always deals with the currently active stack frame. Stack frames are shown horizontally throughout the documentation for space reasons. For example, the active stack frame in the figure above would be shown as:

[ 0 1 2 3 4 5 ]

A value stack index is a signed integer index used in the Duktape API to refer to elements in currently active stack frame, relative to the current frame bottom.

Non-negative (>= 0) indices refer to stack entries in the current stack frame, relative to the frame bottom:

[ 0 1 2 3 4 5! ]

Negative (< 0) indices refer to stack entries relative to the top:

[ -6 -5 -4 -3 -2 -1! ]

The special constant DUK_INVALID_INDEX is a negative integer which denotes an invalid stack index. It can be returned from API calls and can also be given to API calls to indicate a "no value".

The value stack top (or just "top") is the non-negative index of an imaginary element just above the highest used index. For instance, above the highest used index is 5, so the stack top is 6. The top indicates the current stack size, and is also the index of the next element pushed to the stack.

[ 0 1 2 3 4 5! 6? ]

API stack operations are always confined to the current stack frame. There is no way to refer to stack entries below the current frame. This is intentional, as it protects functions in the call stack from affecting each other's values.

Don't confuse with the C stack.

Growing the value stack

At any time, the value stack of a context is allocated for a certain maximum number of entries. An attempt to push values beyond the allocated size will cause an error to be thrown, it will not cause the value stack to be automatically extended. This simplifies the internal implementation and also improves performance by minimizing reallocations when you know, beforehand, that a certain number of entries will be needed during a function.

When a value stack is created or a Duktape/C function is entered, the value stack is always guaranteed to have space for the call arguments and DUK_API_ENTRY_STACK (currently 64) elements. In the typical case this is more than sufficient so that the majority of Duktape/C functions don't need to extend the value stack. Only functions that need more space or perhaps need an input-dependent amount of space need to grow the value stack.

You can extend the stack allocation explicitly with duk_check_stack() or (usually more preferably) duk_require_stack(). Once successfully extended, you are again guaranteed that the specified number of elements can be pushed to the stack. There is no way to shrink the allocation except by returning from a Duktape/C function.

Consider, for instance, the following function which will uppercase an input ASCII string by pushing uppercased characters one-by-one on the stack and then concatenating the result. This example illustrates how the number of value stack entries required may depend on the input (otherwise this is not a very good approach for uppercasing a string):



In addition to user reserved elements, Duktape keeps an automatic internal value stack reserve to ensure all API calls have enough value stack space to work without further allocations. The value stack is also extended and shrunk in somewhat large steps to minimize memory reallocation activity. As a result the internal number of value stack elements available beyond the caller specified extra varies considerably. The caller does not need to take this into account and should never rely on any additional elements being available.

Ecmascript array index

Ecmascript object and array keys can only be strings. Array indices (e.g. 0, 1, 2) are represented as canonical string representations of the respective numbers. More technically, all canonical string representations of the integers in the range [0, 2**32-1] are valid array indices.

To illustrate the Ecmascript array index handling, consider the following example:

var arr = [ 'foo', 'bar', 'quux' ];

print(arr[1]);     // refers to 'bar'
print(arr["1"]);   // refers to 'bar'

print(arr[1.0]);   // refers to 'bar', canonical encoding is "1"
print(arr["1.0"]); // undefined, not an array index

Some API calls operating on Ecmascript arrays accept numeric array index arguments. This is really just a short hand for denoting a string conversion of that number. For instance, if the API is given the integer 123, this really refers to the property name "123".

Internally, Duktape tries to avoid converting numeric indices to actual strings whenever possible, so it is preferable to use array index API calls when they are relevant. Similarly, when writing Ecmascript code it is preferable to use numeric rather than string indices, as the same fast path applies for Ecmascript code.

Duktape API

Duktape API is the collection of user callable API calls defined in duktape.h and documented in the API reference.

The Duktape API calls are generally error tolerant and will check all arguments for errors (such as NULL pointers). However, to minimize footprint, the ctx argument is not checked, and the caller MUST NOT call any Duktape API calls with a NULL context.

All Duktape API calls are potentially macros. Calling code must not rely on any Duktape API call being available as a function pointer. The implementation of a certain API call may change between a macro and an actual function even between compatible releases. The Duktape API provides the following guarantees for macros:

Duktape/C function

A C function with a Duktape/C API signature can be associated with an Ecmascript function object, and gets called when the Ecmascript function object is called. A Duktape/C API function looks as follows:

duk_ret_t my_func(duk_context *ctx) {
    duk_push_int(ctx, 123);
    return 1;
}

The function gets Ecmascript call argument in the value stack of ctx, with duk_get_top() indicating the number of arguments present on the value stack. The this binding is not automatically pushed to the value stack; use duk_push_this() to access it. When creating an Ecmascript function object associated with a Duktape/C function, one can select the desired number of arguments. Extra arguments are dropped and missing arguments are replaced with undefined. A function can also be registered as a vararg function (by giving DUK_VARARGS as the argument count) in which case call arguments are not modified prior to C function entry.

The function can return one of the following:

A negative error return value is intended to simplify common error handling, and is an alternative to constructing and throwing an error explicitly with Duktape API calls. No error message can be given; a message is automatically constructed by Duktape. For example:

duk_ret_t my_func(duk_context *ctx) {
    if (duk_get_top(ctx) == 0) {
        /* throw TypeError if no arguments given */
        return DUK_RET_TYPE_ERROR;
    }
    /* ... */
}

All Duktape/C functions are considered strict in the Ecmascript sense. Duktape API calls always obey Ecmascript strict mode semantics, even when the API calls are made outside of any Duktape/C function, i.e. with an empty call stack. For instance, attempt to delete a non-configurable property using duk_del_prop() will cause an error to be thrown. This is the case with a strict Ecmascript function too:

function f() {
    'use strict';
    var arr = [1, 2, 3];
    return delete arr.length;  // array 'length' is non-configurable
}

print(f());  // this throws an error because f() is strict

Another consequence of Duktape/C function strictness is that the this binding given to Duktape/C functions is not coerced. This is also the case for strict Ecmascript code:

function strictFunc() { 'use strict'; print(typeof this); }
function nonStrictFunc() { print(typeof this); }

strictFunc.call('foo');     // prints 'string' (uncoerced)
nonStrictFunc.call('foo');  // prints 'object' (coerced)

Duktape/C functions are currently always constructable, i.e. they can always be used in new Foo() expressions. You can check whether a function was called in constructor mode as follows:

static duk_ret_t my_func(duk_context *ctx) {
    if (duk_is_constructor_call(ctx)) {
        printf("called as a constructor\n");
    } else {
        printf("called as a function\n");
    }
}

To save memory, Duktape/C functions don't have a prototype property by default, so the default object instance (given to the constructor as this) inherits from Object.prototype. To use a custom prototype you can define prototype for the Duktape/C function explicitly. Another approach is to ignore the default object instance and construct one manually within the Duktape/C call: as with Ecmascript functions, if a constructor returns an object value, that value replaces the default object instance and becomes the value of the new expression.

The this binding is not automatically pushed to the value stack; use duk_push_this() to access it.

Storing state for a Duktape/C function

Sometimes it would be nice to provide parameters or additional state to a Duktape/C function out-of-band, i.e. outside explicit call arguments. There are several ways to achieve this.

Properties of function

First, a Duktape/C function can use its Function object to store state or parameters. A certain Duktape/C function (the actual C function) is always represented by an Ecmascript Function object which is internally associated with the underlying C function. The Function object can be used to store properties related to that particular instance of the function. Note that a certain Duktape/C function can be associated with multiple independent Function objects and thus independent states.

Accessing the Ecmascript Function object related to a Duktape/C function is easy:

duk_push_current_function(ctx);
duk_get_prop_string(ctx, -1, "my_state_variable");

'this' binding

Another alternative for storing state is to call the Duktape/C function as a method and then use the this binding for storing state. For instance, consider a Duktape/C function called as:

foo.my_c_func()

When called, the Duktape/C function gets foo as its this binding, and one could store state directly in foo. The difference to using the Function object approach is that the same object is shared by all methods, which has both advantages and disadvantages.

Accessing the this binding is easy:

duk_push_this(ctx);
duk_get_prop_string(ctx, -1, "my_state_variable");

Hidden Symbol properties

If data needs to be associated with an object, but hidden from Ecmascript code, hidden Symbols can be used as a property key. The key is only accessible via the C API, unless passed to Ecmascript code from the C API. A common use case is to associated backing pointers/data to C memory. Symbols are created as a string, but are differentiated with macros that mark them as Symbols.

For example, setting and getting a hidden symbol on this:

my_context_data_t *my_context_data = malloc(sizeof(my_context_data_t));
duk_push_this(ctx);
duk_push_pointer(ctx, my_context_data);
duk_put_prop_string(ctx, -2, DUK_HIDDEN_SYMBOL("my_context_data"));
/* ... */
duk_push_this(ctx);
duk_get_prop_string(ctx, -1, DUK_HIDDEN_SYMBOL("my_context_data"));
my_context_data_t *my_context_data = duk_get_pointer(ctx, -1);

Magic value of function

Duktape/C function objects can store an internal 16-bit signed integer "magic" value (zero by default) with no extra memory cost. The magic value can be used to pass flags and/or small values to a Duktape/C function at minimal cost, so that a single native function can provide slightly varied behavior for multiple function objects:

/* Magic value example: two lowest bits are used for a prefix index, bit 2 (0x04)
 * is used to select newline style for a log write helper.
 */
const char *prefix[4] = { "INFO", "WARN", "ERROR", "FATAL" };
duk_int_t magic = duk_get_current_magic(ctx);

printf("%s: %s", prefix[magic & 0x03], duk_safe_to_string(ctx, 0));
if (magic & 0x04) {
    printf("\r\n");
} else {
    printf("\n");
}

For an API usage example, see the test case test-get-set-magic.c. Duktape uses magic values a lot internally to minimize size of compiled code, see e.g. duk_bi_math.c.

The magic value mechanism is liable to change between major Duktape versions, as the number of available spare bits changes. Use magic values only when it really matters for footprint. Properties stored on the function object is a more stable alternative.

Heap stash

The heap stash is an object visible only from C code. It is associated with the Duktape heap, and allows Duktape/C code to store "under the hood" state data which is not exposed to Ecmascript code. It is accessed with the duk_push_heap_stash() API call.

Global stash

The global stash is like the heap stash, but is associated with a global object. It is accessed with the duk_push_global_stash() API call. There can be several environments with different global objects within the same heap.

Thread stash

The thread stash is like the heap stash, but is associated with a Duktape thread (i.e. a ctx pointer). It is accessible with the duk_push_thread_stash() API call.

Duktape version specific code

The Duktape version is available through the DUK_VERSION define, with the numeric value (major * 10000) + (minor * 100) + patch. The same value is available to Ecmascript code through Duktape.version. Calling code can use this define for Duktape version specific code.

For C code:

#if (DUK_VERSION >= 10203)
/* Duktape 1.2.3 or later */
#elif (DUK_VERSION >= 800)
/* Duktape 0.8.0 or later */
#else
/* Duktape lower than 0.8.0 */
#endif

For Ecmascript code (also see Duktape built-ins):

if (typeof Duktape !== 'object') {
    print('not Duktape');
} else if (Duktape.version >= 10203) {
    print('Duktape 1.2.3 or higher');
} else if (Duktape.version >= 800) {
    print('Duktape 0.8.0 or higher (but lower than 1.2.3)');
} else {
    print('Duktape lower than 0.8.0');
}

Numeric error codes

When errors are created or thrown using the Duktape API, the caller must assign a numeric error code to the error. Error codes are positive integers, with a range restricted to 24 bits at the moment: the allowed error number range is thus [1,16777215]. Built-in error codes are defined in duktape.h, e.g. DUK_ERR_TYPE_ERROR.

The remaining high bits are used internally to carry e.g. additional flags. Negative error values are used in the Duktape/C API as a shorthand to automatically throw an error.

Error handling

Error handling in the Duktape API is similar to how Ecmascript handles errors: errors are thrown either explicitly or implicitly, then caught and handled. However, instead of a try-catch statement application code uses protected Duktape API calls to establish points in C code where errors can be caught and handled. An uncaught error causes the fatal error handler to be called, which is considered an unrecoverable situation and should ordinarily be avoided, see Normal and fatal errors and How to handle fatal errors.

To avoid fatal errors, typical application code should establish an error catch point before making other Duktape API calls. This is done using protected Duktape API calls, for example:

An example of the first technique:

/* Use duk_peval() variant to evaluate a file so that script errors are
 * handled safely.  Both syntax errors and runtime errors are caught.
 */

push_file_as_string(ctx, "myscript.js");
if (duk_peval(ctx) != 0) {
    /* Use duk_safe_to_string() to convert error into string.  This API
     * call is guaranteed not to throw an error during the coercion.
     */
    printf("Script error: %s\n", duk_safe_to_string(ctx, -1));
}
duk_pop(ctx);

An example of the second technique:

/* Use duk_safe_call() to wrap all unsafe code into a separate C function.
 * This approach has the advantage of covering all API calls automatically
 * but is a bit more verbose.
 */

static duk_ret_t unsafe_code(duk_context *ctx, void *udata) {
    /* Here we can use unprotected calls freely. */

    (void) udata;  /* 'udata' may be used to pass e.g. a struct pointer */

    push_file_as_string(ctx, "myscript.js");
    duk_eval(ctx);

    /* ... */

    return 0;  /* success return, no return value */
}

/* elsewhere: */

if (duk_safe_call(ctx, unsafe_code, NULL /*udata*/, 0 /*nargs*/, 1 /*nrets */) != 0) {
    /* The 'nrets' argument should be at least 1 so that an error value
     * is left on the stack if an error occurs.  To avoid further errors,
     * use duk_safe_to_string() for safe error printing.
     */
    printf("Unexpected error: %s\n", duk_safe_to_string(ctx, -1));
}
duk_pop(ctx);

Even within protected calls there are some rare cases, such as internal errors, that will either cause a fatal error or propagate an error outwards from a protected API call. These should only happen in abnormal conditions and are not considered recoverable. To handle also these cases well, a production quality application should always have a fatal error handler with a reasonable strategy for dealing with fatal errors. Such a strategy is necessarily context dependent, but could be something like:

Note that it may be fine for some applications to make API calls without an error catcher and risk throwing uncaught errors leading to a fatal error. It's not possible to continue execution after a fatal error, so such applications would typically simply exit when a fatal error occurs. Even without an actual recovery strategy, a fatal error handler should be used to e.g. write fatal error information to stderr before process exit.

Normal and fatal errors

An ordinary error is caused by a throw statement, a duk_throw() API call (or similar), or by an internal, recoverable Duktape error. Ordinary errors can be caught with a try-catch in Ecmascript code or e.g. duk_pcall() (see API calls tagged protected) in C code.

A fatal error is caused by an uncaught error, an explicit call to duk_fatal(), or an unrecoverable error inside Duktape. Each Duktape heap has a heap-wide fatal error handler registered in duk_create_heap(); if no handler is given a built-in default handler is used. There's no safe way to resume execution after a fatal error, so that a fatal error handler must not return or call into the Duktape API. Instead the handler should e.g. write out the message to the console or a log file and then exit the process (or restart an embedded device).

Fatal errors may also happen without any heap context, so that Duktape can't look up a possible heap-specific fatal error handler. Duktape will then always call the built-in default fatal error handler (with the handler userdata argument set to NULL). Fatal errors handled this way are currently limited to assertion failures, so that if you don't enable assertions, no such errors will currently occur. All fatal error handling goes through the heap-associated fatal error handler which is in direct application control.

The built-in default fatal error handler will write a debug log message (but won't write anything to stdout to stderr), and will then call abort(); if that fails, it enters an infinite loop to ensure execution doesn't resume after a fatal error. It is strongly recommended to both (1) provide a custom fatal error handler in heap creation and (2) replace the built-in default fatal error handler using the DUK_USE_FATAL_HANDLER() option in duk_config.h.

See How to handle fatal errors for more detail and examples.