Cpp-Taskflow  2.2.0
C6: Manage Threads and Executor

We discuss in this chapter the thread management and task execution schemes in Cpp-Taskflow. We will go through the concept of thread, ownership, and executor in Cpp-Taskflow.

Master, Workers, and Executor

Cpp-Taskflow defines a strict relationship between the master and workers. Master is the thread that creates the executor object and workers are threads that invoke the callable target of a task. Each executor manages its own set of worker threads in a shared pool to schedule tasks. By default, Cpp-Taskflow uses std::thread::hardware_concurrency to decide the number of worker threads and std::thread::get_id to identify the ownership between the master and workers.

std::cout << std::thread::hardware_concurrency() << std::endl; // 8, for example
std::cout << std::thread::get_id() << std::endl; // master thread id
tf::Executor exe1; // create an executor with 8 workers
tf::Executor exe2(4); // create an executor with 4 workers

In the above example, the master thread owns both executor objects. The first executor exe1 creates eight (default by std::thread::hardware_concurrency) worker threads and the second executor exe2 creates four worker threads. Including the master thread, there will be a total of 1 + 8 + 4 = 13 threads running in this program. If you create an executor with zero workers, the master will carry out all the tasks by itself. That is, using one worker and zero worker are conceptually equivalent to each other since they both end up using one thread to run all tasks (see the snippet below).

tf::Executor exe1(0); // one master, zero worker (master to run tasks)
tf::Executor exe2(1); // one master, one worker (one thread to run tasks)

In general, the master thread is where you start the main function (main thread), while the worker threads are transparently maintained by its own executor. Cpp-Taskflow's executor implements a very efficient work-stealing algorithm to schedule the execution of tasks.

Thread Safety

tf::Executor is NOT thread-safe. Touching an executor from multiple threads can result in undefined behavior. Notice that this is different from running multiple taskflows on a same executor which is valid. Thread safety has nothing to do with the master nor the workers. It is completely safe to access an executor as long as only one thread presents at a time. However, we strongly recommend users to acknowledge the definition of the master and the workers, and separate the program control flow accordingly. Having a clear thread ownership can greatly reduce the chance of buggy implementations and undefined behaviors.

Monitor Thread Activities

Inspecting the thread activities is very important for performance analysis. It allows you to know when each task starts and ends participating in the task scheduling. Cpp-Taskflow provides a default observer class tf::ExecutorObserver for this purpose. The following example shows how to create an observer from an executor.

Note that each executor can only have an observer at a time. An observer will automatically record the start and end timestamps of each executed task. Users can query, dump or remove the timestamps through the tf::ExecutorObserver::num_tasks, tf::ExecutorObserver::dump and tf::ExecutorObserver::clear methods.

1. tf::Executor executor;
3.
4. executor.run(taskflow).get(); // do something
5.
6. // Query the total number of tasks (number of timestamp pairs)
7. auto num_tasks = observer->num_tasks();
8.
9. // Dump the timeline data in JSON format
10. std::string timelines_in_json = observer->dump();
11.
12. // Clear the timeline data
13. observer->clear();

Debrief:

  • Line 2-4 creates an observer and a task dependency graph with four tasks and dispatch the tasks to execution.
  • Line 7 query the total number of tasks (number of timestamp pair) through observer
  • Line 10 dump the timestamps to a std::string in JSON format
  • Line 13 remove all timestamps in the observer

You can visualize the timeline data in a Chrome browser:

  • Step 1: save the JSON timeline data to a file
  • Step 2: launch the Chrome browser and open a tab with the url: chrome://tracing
  • Step 3: load the JSON file
timeline.png

Tasks will be categorized by the executing thread and each task is named with i_j where i is the thread id and j is the task number. You can pan or zoom in/out the timeline to get a detailed view.

Customize Your Own Observer

You can derive your own observer from the base interface class tf::ExecutorObserverInterface to customize the observing methods.