HPX - High Performance ParalleX


HPX Command Line Options

More Details about HPX Command Line Options

The predefined command line options for any application using hpx::init are described in the table below:

Table 13. Default HPX Command Line Options



HPX options (allowed on command line only)


--hpx:help, --help or -h

print out program usage (default: this message), possible values: 'full' (additionally prints options from components)

--hpx:version, --version or -v

print out HPX version and copyright information


print out HPX configuration information

--hpx:options-file arg

specify a file containing command line options (alternatively: @filepath)

HPX options (additionally allowed in an options file)



run this instance in worker mode


run this instance in console mode


run this instance in worker mode, but connecting late


run AGAS server as part of this runtime instance


run the hpx_main function, regardless of locality mode

--hpx:hpx arg

the IP address the HPX parcelport is listening on, expected format: 'address:port' (default:

--hpx:agas arg

the IP address the AGAS root server is running on, expected format: 'address:port' (default:


run only the AGAS server

--hpx:nodefile arg

the file name of a node file to use (list of nodes, one node name per line and core)

--hpx:nodes arg

the (space separated) list of the nodes to use (usually this is extracted from a node file)


this can be used to end the list of nodes specified using the option --hpx:nodes

--hpx:ifsuffix arg

suffix to append to host names in order to resolve them to the proper network interconnect

--hpx:ifprefix arg

prefix to prepend to host names in order to resolve them to the proper network interconnect

--hpx:iftransform arg

sed-style search and replace (s/search/replace/) used to transform host names to the proper network interconnect

--hpx:localities arg

the number of localities to wait for at application startup (default: 1)

--hpx:node arg

number of the node this locality is run on (must be unique)


the first processing unit this instance of HPX should be run on (default: 0), valid for --hpx:queuing=local, --hpx:queuing=abp-priority, --hpx:queuing=static, and --hpx:queuing=local-priority only


the step between used processing unit numbers for this instance of HPX (default: 1), valid for --hpx:queuing=local, --hpx:queuing=abp-priority, --hpx:queuing=static and --hpx:queuing=local-priority only

--hpx:threads arg

the number of operating system threads to spawn for this HPX locality (default: 1, using 'all' will spawn one thread for each processing unit)

--hpx:cores arg

the number of cores to utilize for this HPX locality locality (default: 'all', i.e. the number of cores is based on the number of threads (--hpx:threads) assuming --hpx:bind=compact)

--hpx:affinity arg

the affinity domain the OS threads will be confined to, possible values: pu, core, numa, machine (default: pu), valid for --hpx:queuing=local, --hpx:queuing=abp-priority, --hpx:queuing=static, and --hpx:queuing=local-priority only

--hpx:bind arg

the detailed affinity description for the OS threads, see the additional documentation for a detailed description of possible values. Do not use with --hpx:pu-step, --hpx:pu-offset, or --hpx:affinity options. Implies --hpx:numa-sensitive.


print to the console the bit masks calculated from the arguments specified to all --hpx:bind options.

--hpx:queuing arg

the queue scheduling policy to use, options are 'local/l', 'local-priority/lo', 'abp/a', 'abp-priority', 'hierarchy/h', and 'periodic/pe' (default: local-priority/lo)


the arity of the of the thread queue tree, valid for --hpx:queuing=hierarchy only (default: 2)

--hpx:high-priority-threads arg

the number of operating system threads maintaining a high priority queue (default: number of OS threads), valid for --hpx:queuing=local, --hpx:queuing=abp-priority, and --hpx:queuing=local-priority only


makes the local-priority scheduler NUMA sensitive, valid for --hpx:queuing=local, --hpx:queuing=abp-priority, --hpx:queuing=static, and --hpx:queuing=local-priority only

HPX configuration options


--hpx:app-config arg

load the specified application configuration (ini) file

--hpx:config arg

load the specified hpx configuration (ini) file

--hpx:ini arg

add a configuration definition to the default runtime configuration


exit after configuring the runtime

HPX debugging options



list all registered symbolic names after startup


list all dynamic component types after startup


print the initial runtime configuration


print the final runtime configuration


enable all messages on the HPX log channel and send all HPX logs to the target destination


enable all messages on the AGAS log channel and send all AGAS logs to the target destination


debug command line processing

--hpx:attach-debugger arg

wait for a debugger to be attached, possible arg values: startup or exception (default: startup)

HPX options related to performance counters



print the specified performance counter either repeatedly or before shutting down the system (see option --hpx:print-counter-interval)


print the performance counter(s) specified with --hpx:print-counter repeatedly after the time interval (specified in milliseconds) (default: 0, which means print once at shutdown)


print the performance counter(s) specified with --hpx:print-counter to the given file (default: console)


list the names of all registered performance counters, possible values: 'minimal' (prints counter name skeletons), 'full' (prints all available counter names)


list the description of all registered performance counters, possible values: 'minimal' (prints info for counter name skeletons), 'full' (prints all available counter infos)

Command Line Argument Shortcuts

Additionally, the following shortcuts are available from every HPX application.

Table 14. Predefined command line option shortcuts

Shortcut option

Equivalent long option













































It is possible to define your own shortcut options. In fact, all of the shortcuts listed above are pre-defined using the technique described here. In fact, it is possible to redefine any of the pre-defined shortcuts to expand differently as well.

Shortcut options are obtained from the internal configuration database. They are stored as key-value properties in a special properties section named hpx.commandline. You can define your own shortcuts by adding the corresponding definitions to one of the ini configuration files as described in the section Configure HPX Applications. For instance, in order to define a command line shortcut --pc which should expand to --hpx:print-counter, the following configuration information needs to be added to one of the ini configuration files:

--pc = --hpx:print-counter
[Note] Note

Any arguments for shortcut options passed on the command line are retained and passed as arguments to the corresponding expanded option. For instance, given the definition above, the command line option


would be expanded to


[Important] Important

Any shortcut option should either start with a single '-' or with two '--' characters. Shortcuts starting with a single '-' are interpreted as short options (i.e. everything after the first character following the '-' is treated as the argument). Shortcuts starting with '--' are interpreted as long options. No other shortcut formats are supported.

Specifying Options for Single Localities Only

For runs involving more than one locality it is sometimes desirable to supply specific command line options to single localities only. When the HPX application is launched using a scheduler (like PBS, for more details see section Using PBS), specifying dedicated command line options for single localities may be desirable. For this reason all of the command line options which have the general format --hpx:<some_key> can be used in a more general form: --hpx:<N>:<some_key>, where <N> is the number of the locality this command line options will be applied to, all other localities will simply ignore the option. For instance, the following PBS script passes the option --hpx:pu-offset=4 to the locality '1' only.

#PBS -l nodes=2:ppn=4


pbsdsh -u $APP_PATH $APP_OPTIONS --hpx:1:pu-offset=4 --hpx:nodes=`cat $PBS_NODEFILE`
[Caution] Caution

If the first application specific argument (inside $APP_OPTIONS) is a non-option (i.e. does not start with a '-' or a '--', then it must be placed before the option --hpx:nodes, which, in this case, should be the last option on the command line.

Alternatively, use the option --hpx:endnodes to explicitly mark the end of the list of node names:

pbsdsh -u $APP_PATH --hpx:1:pu-offset=4 --hpx:nodes=`cat $PBS_NODEFILE` --hpx:endnodes $APP_OPTIONS