HPX - High Performance ParalleX

PrevUpHomeNext
Built-in Default Configuration Settings

During startup any HPX application applies a predefined search pattern to locate one or more configuration files. All found files will be read and merged in the sequence they are found into one single internal data structure holding all configuration properties.

As a first step the internal configuration database is filled with a set of default configuration properties. Those settings are described on a section by section basis below.

[Note] Note

You can print the default configuration settings used for an executable by specifying the command line option --hpx:dump-config.

The system Configuration Section

[system]
pid = <process-id>
prefix = <current prefix path of core HPX library>
executable = <current prefix path of executable>

Property

Description

system.pid

This is initialized to store the current OS-process id of the application instance.

system.prefix

This is initialized to the base directory HPX has been loaded from.

system.executable_prefix

This is initialized to the base directory the current executable has been loaded from.

The hpx Configuration Section

[hpx]
location = ${HPX_LOCATION:$[system.prefix]}
component_path = $[hpx.location]/lib/hpx:$[system.executable_prefix]/lib/hpx:$[system.executable_prefix]/../lib/hpx
master_ini_path = $[hpx.location]/share/hpx-<version>:$[system.executable_prefix]/share/hpx-<version>:$[system.executable_prefix]/../share/hpx-<version>
ini_path = $[hpx.master_ini_path]/ini
os_threads = 1
localities = 1
program_name =
cmd_line =
lock_detection = 0
minimal_deadlock_detection = <debug>

[hpx.stacks]
small_size = ${HPX_SMALL_STACK_SIZE:<hpx_small_stack_size>}
medium_size = ${HPX_MEDIUM_STACK_SIZE:<hpx_medium_stack_size>}
large_size = ${HPX_LARGE_STACK_SIZE:<hpx_large_stack_size>}
huge_size = ${HPX_HUGE_STACK_SIZE:<hpx_huge_stack_size>}
use_guard_pages = ${HPX_USE_GUARD_PAGES:1}

Property

Description

hpx.location

This is initialized to the id of the locality this application instance is running on.

hpx.component_path

This is initialized to the list of directories where the HPX runtime library will look for installed components. Duplicates are discarded. This property can refer to a list of directories separated by ':' (Linux, Android, and MacOS) or using ';' (Windows).

hpx.master_ini_path

This is initialized to the list of default paths of the main hpx.ini configuration files. This property can refer to a list of directories separated by ':' (Linux, Android, and MacOS) or using ';' (Windows).

hpx.ini_path

This is initialized to the default path where HPX will look for more ini configuration files. This property can refer to a list of directories separated by ':' (Linux, Android, and MacOS) or using ';' (Windows).

hpx.os_threads

This setting reflects the number of OS-threads used for running HPX-threads. Defaults to 1.

hpx.localities

This setting reflects the number of localities the application is running on. Defaults to 1.

hpx.program_name

This setting reflects the program name of the application instance. Initialized from the command line (argv\[0\]).

hpx.cmd_line

This setting reflects the actual command line used to launch this application instance.

hpx.lock_detection

This setting verifies that no locks are being held while a HPX thread is suspended. This setting is applicable only if HPX_VERIFY_LOCKS is set during configuration in CMake.

hpx.minimal_deadlock_detection

This setting enables support for minimal deadlock detection for HPX-threads. By default this is set to 1 (for Debug builds) or to 0 (for Release, RelWithDebInfo, RelMinSize builds), this setting is effective only if HPX_THREAD_MINIMAL_DEADLOCK_DETECTION is set during configuration in CMake.

hpx.stacks.small_size

This is initialized to the small stack size to be used by HPX-threads. Set by default to the value of the compile time preprocessor constant HPX_SMALL_STACK_SIZE (defaults to 0x8000).

hpx.stacks.medium_size

This is initialized to the medium stack size to be used by HPX-threads. Set by default to the value of the compile time preprocessor constant HPX_MEDIUM_STACK_SIZE (defaults to 0x20000).

hpx.stacks.large_size

This is initialized to the large stack size to be used by HPX-threads. Set by default to the value of the compile time preprocessor constant HPX_LARGE_STACK_SIZE (defaults to 0x200000).

hpx.stacks.huge_size

This is initialized to the huge stack size to be used by HPX-threads. Set by default to the value of the compile time preprocessor constant HPX_HUGE_STACK_SIZE (defaults to 0x2000000).

hpx.stacks.use_guard_pages

This entry controls whether the coroutine library will generate stack guard pages or not. This entry is applicable on Linux only and only if the HPX_USE_GENERIC_COROUTINE_CONTEXT option is not enabled and the HPX_THREAD_GUARD_PAGE is set to 1 while configuring the build system. It is set by default to 1.

The hpx.threadpools Configuration Section

[hpx.threadpools]
io_pool_size = ${HPX_NUM_IO_POOL_THREADS:2}
parcel_pool_size = ${HPX_NUM_PARCEL_POOL_THREADS:2}
timer_pool_size = ${HPX_NUM_TIMER_POOL_THREADS:2}

Property

Description

hpx.threadpools.io_pool_size

The value of this property defines the number of OS-threads created for the internal I/O thread pool.

hpx.threadpools.parcel_pool_size

The value of this property defines the number of OS-threads created for the internal parcel thread pool.

hpx.threadpools.timer_pool_size

The value of this property defines the number of OS-threads created for the internal timer thread pool.

The hpx.components Configuration Section

[hpx.components]
load_external = ${HPX_LOAD_EXTERNAL_COMPONENTS:1}

Property

Description

hpx.components.load_external

This entry defines whether external components will be loaded on this locality. This entry normally is set to 1 and usually there is no need to directly change this value. It is automatically set to 0 for a dedicated AGAS server locality.

Additionally, the section hpx.components will be populated with the information gathered from all found components. The information loaded for each of the components will contain at least the following properties:

[hpx.components.<component_instance_name>]
name = <component_name>
path = <full_path_of_the_component_module>
enabled = $[hpx.components.load_external]

Property

Description

hpx.components.<component_instance_name>.name

This is the name of a component, usually the same as the second argument to the macro used while registering the component with HPX_REGISTER_MINIMAL_COMPONENT_FACTORY. Set by the component factory.

hpx.components.<component_instance_name>.path

This is either the full path file name of the component module or the directory the component module is located in. In this case, the component module name will be derived from the property hpx.components.<component_instance_name>.name. Set by the component factory.

hpx.components.<component_instance_name>.enabled

This setting explicitly enables or disables the component. This is an optional property, HPX assumed that the component is enabled if it is not defined.

The value for <component_instance_name> is usually the same as for the corresponding name property. However generally it can be defined to any arbitrary instance name. It is used to distinguish between different ini sections, one for each component.

The hpx.parcel Configuration Section

[hpx.parcel]
address = ${HPX_PARCEL_SERVER_ADDRESS:<hpx_initial_ip_address>}
port = ${HPX_PARCEL_SERVER_PORT:<hpx_initial_ip_port>}
bootstrap = ${HPX_PARCEL_BOOTSTRAP:<hpx_parcel_bootstrap>}
max_connections = ${HPX_PARCEL_MAX_CONNECTIONS:<hpx_parcel_max_connections>}
max_connections_per_locality = ${HPX_PARCEL_MAX_CONNECTIONS_PER_LOCALITY:<hpx_parcel_max_connections_per_locality>}
max_message_size = ${HPX_PARCEL_MAX_MESSAGE_SIZE:<hpx_parcel_max_message_size>}
array_optimization = ${HPX_PARCEL_ARRAY_OPTIMIZATION:1}
zero_copy_optimization = ${HPX_PARCEL_ZERO_COPY_OPTIMIZATION:$[hpx.parcel.array_optimization]}

Property

Description

hpx.parcel.address

This property defines the default IP address to be used for the parcel layer to listen to. This IP address will be used as long as no other values are specified (for instance using the --hpx:hpx command line option). The expected format is any valid IP address or domain name format which can be resolved into an IP address. The default depends on the compile time preprocessor constant HPX_INITIAL_IP_ADDRESS ("127.0.0.1").

hpx.parcel.port

This property defines the default IP port to be used for the parcel layer to listen to. This IP port will be used as long as no other values are specified (for instance using the --hpx:hpx command line option). The default depends on the compile time preprocessor constant HPX_INITIAL_IP_PORT (7010).

hpx.parcel.bootstrap

This property defines which parcelport type should be used during application bootstrap. The default depends on the compile time preprocessor constant HPX_PARCEL_BOOTSTRAP ("tcpip").

hpx.parcel.max_connections

This property defines how many network connections between different localities are overall kept alive by each of locality. The default depends on the compile time preprocessor constant HPX_PARCEL_MAX_CONNECTIONS (512).

hpx.parcel.max_connections_per_locality

This property defines the maximum number of network connections that one locality will open to another locality. The default depends on the compile time preprocessor constant HPX_PARCEL_MAX_CONNECTIONS_PER_LOCALITY (4).

hpx.parcel.max_message_size

This property defines the maximum allowed message size which will be transferrable through the parcel layer. The default depends on the compile time preprocessor constant HPX_PARCEL_MAX_MESSAGE_SIZE (1000000000) bytes.

hpx.parcel.array_optimization

This property defines whether this locality is allowed to utilize array optimizations during serialization of parcel data. The default is 1.

hpx.parcel.zero_copy_optimization

This property defines whether this locality is allowed to utilize zero copy optimizations during serialization of parcel data. The default is the same value as set for hpx.parcel.array_optimization.

[hpx.parcel.tcpip]
enable = ${HPX_HAVE_PARCELPORT_TCPIP:1}
array_optimization = ${HPX_PARCEL_TCPIP_ARRAY_OPTIMIZATION:$[hpx.parcel.array_optimization]}
zero_copy_optimization = ${HPX_PARCEL_TCPIP_ZERO_COPY_OPTIMIZATION:$[hpx.parcel.zero_copy_optimization]}

Property

Description

hpx.parcel.tcpip.enable

Enable the use of the default TCPIP parcelport. Note that the initial bootstrap of the overall HPX application will be performed using the default TCPIP connections. This parcelport is enabled by default. This will be disabled only if MPI is enabled (see below).

hpx.parcel.tcpip.array_optimization

This property defines whether this locality is allowed to utilize array optimizations in the TCP/IP parcelport during serialization of parcel data. The default is the same value as set for hpx.parcel.array_optimization.

hpx.parcel.tcpip.zero_copy_optimization

This property defines whether this locality is allowed to utilize zero copy optimizations in the TC{/IP parcelport during serialization of parcel data. The default is the same value as set for hpx.parcel.zero_copy_optimization.

The following settings relate to the shared memory parcelport (which is usable for communication between two localities on the same node). These settings take effect only if the compile time constant HPX_HAVE_PARCELPORT_SHMEM is set (the equivalent cmake variable is HPX_HAVE_PARCELPORT_SHMEM, and has to be set to ON).

[hpx.parcel.shmem]
enable = ${HPX_HAVE_PARCELPORT_SHMEM:0}
data_buffer_cache_size=${HPX_PARCEL_SHMEM_DATA_BUFFER_CACHE_SIZE:512}
array_optimization = ${HPX_PARCEL_SHMEM_ARRAY_OPTIMIZATION:$[hpx.parcel.array_optimization]}

Property

Description

hpx.parcel.shmem.enable

Enable the use of the shared memory parcelport for connections between localities running on the same node. Note that the initial bootstrap of the overall HPX application will still be performed using the default TCP connections. This parcelport is disabled by default.

hpx.parcel.shmem.data_buffer_cache_size

This property specifies the number of cached data buffers used for interprocess communication between localities on the same node. The default depends on the compile time preprocessor constant HPX_PARCEL_SHMEM_DATA_BUFFER_CACHE_SIZE (512).

hpx.parcel.shmem.array_optimization

This property defines whether this locality is allowed to utilize array optimizations in the shared memory parcelport during serialization of parcel data. The default is the same value as set for hpx.parcel.array_optimization.

The following settings relate to the Infiniband parcelport. These settings take effect only if the compile time constant HPX_HAVE_PARCELPORT_IBVERBS is set (the equivalent cmake variable is HPX_HAVE_PARCELPORT_IBVERBS, and has to be set to ON).

[hpx.parcel.ibverbs]
enable = ${HPX_HAVE_PARCELPORT_IBVERBS:0}
buffer_size = ${HPX_PARCEL_IBVERBS_BUFFER_SIZE:65536}
array_optimization = ${HPX_PARCEL_IBVERBS_ARRAY_OPTIMIZATION:$[hpx.parcel.array_optimization]}

Property

Description

hpx.parcel.ibverbs.enable

Enable the use of the ibverbs parcelport for connections between localities running on a node with infiniband capable hardware. Note that the initial bootstrap of the overall HPX application will still be performed using the default TCP connections. This parcelport is disabled by default.

hpx.parcel.ibverbs.buffer_size

This property specifies the size in bytes of the buffers registered to the infiniband hardware. Parcels which are smaller than this will be serialized and sent over the network in a zero-copy fashion. Parcels bigger than this will be transparently copied to a big enough temporary buffer.

hpx.parcel.ibverbs.array_optimization

This property defines whether this locality is allowed to utilize array optimizations in the ibverbs parcelport during serialization of parcel data. The default is the same value as set for hpx.parcel.array_optimization.

The following settings relate to the MPI parcelport. These settings take effect only if the compile time constant HPX_HAVE_PARCELPORT_MPI is set (the equivalent cmake variable is HPX_HAVE_PARCELPORT_MPI, and has to be set to ON).

[hpx.parcel.mpi]
enable = ${HPX_HAVE_PARCELPORT_MPI:1}
env = ${HPX_PARCELPORT_MPI_ENV:PMI_RANK,OMPI_COMM_WORLD_SIZE}
multithreaded = ${HPX_PARCELPORT_MPI_MULTITHREADED:0}
rank = <MPI_rank>
processor_name = <MPI_processor_name>
array_optimization = ${HPX_PARCEL_MPI_ARRAY_OPTIMIZATION:$[hpx.parcel.array_optimization]}

Property

Description

hpx.parcel.mpi.enable

Enable the use of the MPI parcelport. HPX tries to detect if the application was started within a parallel MPI environment. If the detection was succesful, the MPI parcelport is enabled by default. To explicitly disable the MPI parcelport, set to 0. Note that the initial bootstrap of the overall HPX application will be performed using MPI as well.

hpx.parcel.mpi.env

This property influences which environment variables (comma separated) will be analyzed to find out whether the application was invoked by MPI.

hpx.parcel.mpi.multithreaded

This property is used to determine what threading mode to use when initializing MPI. If this setting is 0, HPX will initialize MPI with MPI_THREAD_SINGLE, if the value is not equal to 0 HPX will will initialize MPI with MPI_THREAD_MULTI.

hpx.parcel.mpi.rank

This property will be initialized to the MPI rank of the locality.

hpx.parcel.mpi.processor_name

This property will be initialized to the MPI processor name of the locality.

hpx.parcel.mpi.array_optimization

This property defines whether this locality is allowed to utilize array optimizations in the MPI parcelport during serialization of parcel data. The default is the same value as set for hpx.parcel.array_optimization.

The hpx.agas Configuration Section

[hpx.agas]
address = ${HPX_AGAS_SERVER_ADDRESS:<hpx_initial_ip_address>}
port = ${HPX_AGAS_SERVER_PORT:<hpx_initial_ip_port>}
service_mode = hosted
dedicated_server = 0
max_pending_refcnt_requests = ${HPX_AGAS_MAX_PENDING_REFCNT_REQUESTS:<hpx_initial_agas_max_pending_refcnt_requests>}
use_caching = ${HPX_AGAS_USE_CACHING:1}
use_range_caching = ${HPX_AGAS_USE_RANGE_CACHING:1}
local_cache_size = ${HPX_AGAS_LOCAL_CACHE_SIZE:<hpx_initial_agas_local_cache_size>}
local_cache_size_per_thread = ${HPX_AGAS_LOCAL_CACHE_SIZE_PER_THREAD:<hpx_initial_agas_local_cache_size_per_thread>}

Property

Description

hpx.agas.address

This property defines the default IP address to be used for the AGAS root server. This IP address will be used as long as no other values are specified (for instance using the --hpx:agas command line option). The expected format is any valid IP address or domain name format which can be resolved into an IP address. The default depends on the compile time preprocessor constant HPX_INITIAL_IP_ADDRESS ("127.0.0.1").

hpx.agas.port

This property defines the default IP port to be used for the AGAS root server. This IP port will be used as long as no other values are specified (for instance using the --hpx:agas command line option). The default depends on the compile time preprocessor constant HPX_INITIAL_IP_PORT (7010).

hpx.agas.service_mode

This property specifies what type of AGAS service is running on this locality. Currently, two modes exist. The locality that acts as the AGAS server runs in bootstrap mode. All other localities are in hosted mode.

hpx.agas.dedicated_server

This property specifies whether the AGAS server is exclusively running AGAS services and not hosting any application components. It is a boolean value. Set to 1 if --hpx-run-agas-server-only is present.

hpx.agas.max_pending_refcnt_requests

This property defines the number of reference counting requests (increments or decrements) to buffer. The default depends on the compile time preprocessor constant HPX_INITIAL_AGAS_MAX_PENDING_REFCNT_REQUESTS (4096).

hpx.agas.use_caching

This property specifies whether a software address translation cache is used. It is a boolean value. Defaults to 1.

hpx.agas.use_range_caching

This property specifies whether range-based caching is used by the software address translation cache. This property is ignored if hpx.agas.use_caching is false. It is a boolean value. Defaults to 1.

hpx.agas.local_cache_size

This property defines the size of the software address translation cache for AGAS services. This property is ignored if hpx.agas.use_caching is false. Note that if hpx.agas.use_range_caching is true, this size will refer to the maximum number of ranges stored in the cache, not the number of entries spanned by the cache. Note also, that the size of the cache will be determined by the larger of the following two numbers: the value of this key and the value of hpx.agas.local_cache_size_per_thread multiplied by the number of threads used systemwide in the running application. The default depends on the compile time preprocessor constant HPX_INITIAL_AGAS_LOCAL_CACHE_SIZE (256).

hpx.agas.local_cache_size_per_thread

This property defines the size of the software address translation cache for AGAS services on a per node basis. This property is ignored if hpx.agas.use_caching is false. Note that if hpx.agas.use_range_caching is true, this size will refer to the maximum number of ranges stored in the cache, not the number of entries spanned by the cache. Note also, that the size of the cache will be determined by the larger of the following two numbers: the hpx.agas.local_cache_size and the value of this key multiplied by the number of threads used system wide in the running application. The default depends on the compile time preprocessor constant HPX_AGAS_LOCAL_CACHE_SIZE_PER_THREAD (32).

The hpx.commandline Configuration Section

The following table lists the definition of all pre-defined command line option shortcuts. For more information about commandline options see the section HPX Command Line Options.

[hpx.commandline]
-a = --hpx:agas
-c = --hpx:console
-h = --hpx:help
--help = --hpx:help
-I = --hpx:ini
-l = --hpx:localities
-p = --hpx:app-config
-q = --hpx:queuing",
-r = --hpx:run-agas-server
-t = --hpx:threads
-v = --hpx:version
--version = --hpx:version
-w = --hpx:worker
-x = --hpx:hpx
-0 = --hpx:node=0
-1 = --hpx:node=1
-2 = --hpx:node=2
-3 = --hpx:node=3
-4 = --hpx:node=4
-5 = --hpx:node=5
-6 = --hpx:node=6
-7 = --hpx:node=7
-8 = --hpx:node=8
-9 = --hpx:node=9

Property

Description

hpx.commandline.-a

On the commandline, -a expands to: --hpx:agas

hpx.commandline.-c

On the commandline, -c expands to: --hpx:console

hpx.commandline.-h

On the commandline, -h expands to: --hpx:help

hpx.commandline.--help

On the commandline, --help expands to: --hpx:help

hpx.commandline.-I

On the commandline, -I expands to: --hpx:ini

hpx.commandline.-l

On the commandline, -l expands to: --hpx:localities

hpx.commandline.-p

On the commandline, -p expands to: --hpx:app-config

hpx.commandline.-q

On the commandline, -q expands to: --hpx:queuing

hpx.commandline.-r

On the commandline, -r expands to: --hpx:run-agas-server

hpx.commandline.-t

On the commandline, -t expands to: --hpx:threads

hpx.commandline.-v

On the commandline, -v expands to: --hpx:version

hpx.commandline.--version

On the commandline, --version expands to: --hpx:version

hpx.commandline.-w

On the commandline, -w expands to: --hpx:worker

hpx.commandline.-x

On the commandline, -x expands to: --hpx:hpx

hpx.commandline.-0

On the commandline, -0 expands to: --hpx:node=0

hpx.commandline.-1

On the commandline, -1 expands to: --hpx:node=1

hpx.commandline.-2

On the commandline, -2 expands to: --hpx:node=2

hpx.commandline.-3

On the commandline, -3 expands to: --hpx:node=3

hpx.commandline.-4

On the commandline, -4 expands to: --hpx:node=4

hpx.commandline.-5

On the commandline, -5 expands to: --hpx:node=5

hpx.commandline.-6

On the commandline, -6 expands to: --hpx:node=6

hpx.commandline.-7

On the commandline, -7 expands to: --hpx:node=7

hpx.commandline.-8

On the commandline, -8 expands to: --hpx:node=8

hpx.commandline.-9

On the commandline, -9 expands to: --hpx:node=9


PrevUpHomeNext