org.apache.tez.dag.api
Class TezConfiguration

java.lang.Object
  org.apache.hadoop.conf.Configuration
      org.apache.tez.dag.api.TezConfiguration

All Implemented Interfaces:

Iterable<Map.Entry<String,String>>, org.apache.hadoop.io.Writable

public class TezConfiguration
extends org.apache.hadoop.conf.Configuration

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.conf.Configuration

org.apache.hadoop.conf.Configuration.IntegerRanges

Field Summary

static String	DAG_RECOVERY_DATA_DIR_NAME
static String	DAG_RECOVERY_ENABLED
static boolean	DAG_RECOVERY_ENABLED_DEFAULT
static String	DAG_RECOVERY_FILE_IO_BUFFER_SIZE
static int	DAG_RECOVERY_FILE_IO_BUFFER_SIZE_DEFAULT
static String	DAG_RECOVERY_FLUSH_INTERVAL_SECS
static int	DAG_RECOVERY_FLUSH_INTERVAL_SECS_DEFAULT
static String	DAG_RECOVERY_MAX_UNFLUSHED_EVENTS
static int	DAG_RECOVERY_MAX_UNFLUSHED_EVENTS_DEFAULT
static String	DAG_RECOVERY_RECOVER_FILE_SUFFIX
static String	DAG_RECOVERY_SUMMARY_FILE_SUFFIX
static String	TASK_HEARTBEAT_TIMEOUT_MS
static String	TASK_TIMEOUT
static String	TEZ_AM_CANCEL_DELEGATION_TOKEN
static boolean	TEZ_AM_CANCEL_DELEGATION_TOKEN_DEFAULT
static String	TEZ_AM_CLIENT_AM_PORT_RANGE Range of ports that the AM can use when binding.
static String	TEZ_AM_CLIENT_THREAD_COUNT Number of threads to handle job client RPC requests.
static int	TEZ_AM_CLIENT_THREAD_COUNT_DEFAULT
static String	TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS
static boolean	TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS_DEFAULT
static String	TEZ_AM_CONTAINER_REUSE_ENABLED Configuration to specify whether container should be reused.
static boolean	TEZ_AM_CONTAINER_REUSE_ENABLED_DEFAULT
static String	TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS The amount of time to wait before assigning a container to the next level of locality.
static long	TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS_DEFAULT
static String	TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED Whether to reuse containers for non-local tasks.
static boolean	TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED_DEFAULT
static String	TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED Whether to reuse containers for rack local tasks.
static boolean	TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED_DEFAULT
static String	TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS The amount of time to hold on to a container if no task can be assigned to it immediately.
static long	TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS_DEFAULT
static String	TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT Upper limit on the number of threads user to launch containers in the app master.
static int	TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT_DEFAULT
static String	TEZ_AM_ENV User-provided env for the Tez AM.
static String	TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION
static float	TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION_DEFAULT
static String	TEZ_AM_GROUPING_SPLIT_BY_COUNT
static boolean	TEZ_AM_GROUPING_SPLIT_BY_COUNT_DEFAULT
static String	TEZ_AM_GROUPING_SPLIT_BY_LENGTH
static boolean	TEZ_AM_GROUPING_SPLIT_BY_LENGTH_DEFAULT
static String	TEZ_AM_GROUPING_SPLIT_COUNT
static String	TEZ_AM_GROUPING_SPLIT_MAX_SIZE Upper bound on the size (in bytes) of a grouped split, to avoid generating excessively large splits.
static long	TEZ_AM_GROUPING_SPLIT_MAX_SIZE_DEFAULT
static String	TEZ_AM_GROUPING_SPLIT_MIN_SIZE Lower bound on the size (in bytes) of a grouped split, to avoid generating too many splits.
static long	TEZ_AM_GROUPING_SPLIT_MIN_SIZE_DEFAULT
static String	TEZ_AM_GROUPING_SPLIT_WAVES The multiplier for available queue capacity when determining number of tasks for a Vertex.
static float	TEZ_AM_GROUPING_SPLIT_WAVES_DEFAULT
static String	TEZ_AM_INLINE_TASK_EXECUTION_ENABLED Tez AM Inline Mode flag.
static boolean	TEZ_AM_INLINE_TASK_EXECUTION_ENABLED_DEFAULT Tez AM Inline Mode flag.
static String	TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS The maximium number of tasks running in parallel in inline mode.
static int	TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS_DEFAULT The maximium number of tasks running in parallel in inline mode.
static String	TEZ_AM_JAVA_OPTS Java options for the Tez AppMaster process.
static String	TEZ_AM_JAVA_OPTS_DEFAULT
static String	TEZ_AM_LOG_LEVEL Root Logging level passed to the Tez app master.
static String	TEZ_AM_LOG_LEVEL_DEFAULT
static String	TEZ_AM_MAX_APP_ATTEMPTS
static int	TEZ_AM_MAX_APP_ATTEMPTS_DEFAULT
static String	TEZ_AM_MAX_TASK_ATTEMPTS
static int	TEZ_AM_MAX_TASK_ATTEMPTS_DEFAULT
static String	TEZ_AM_MAX_TASK_FAILURES_PER_NODE
static int	TEZ_AM_MAX_TASK_FAILURES_PER_NODE_DEFAULT
static String	TEZ_AM_NODE_BLACKLISTING_ENABLED
static boolean	TEZ_AM_NODE_BLACKLISTING_ENABLED_DEFAULT
static String	TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD
static int	TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD_DEFAULT
static String	TEZ_AM_PLAN_REMOTE_PATH The complete path to the serialized dag plan file TEZ_AM_PLAN_PB_BINARY.
static String	TEZ_AM_PREFIX
static String	TEZ_AM_RESOURCE_CPU_VCORES
static int	TEZ_AM_RESOURCE_CPU_VCORES_DEFAULT
static String	TEZ_AM_RESOURCE_MEMORY_MB The amount of memory to be used by the AppMaster
static int	TEZ_AM_RESOURCE_MEMORY_MB_DEFAULT
static String	TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX The maximum heartbeat interval between the AM and RM in milliseconds
static int	TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX_DEFAULT
static String	TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_CLIENT
static String	TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL
static String	TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION
static float	TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION_DEFAULT
static String	TEZ_AM_STAGING_DIR The staging dir used while submitting DAGs
static String	TEZ_AM_STAGING_DIR_DEFAULT
static String	TEZ_AM_TASK_LISTENER_THREAD_COUNT
static int	TEZ_AM_TASK_LISTENER_THREAD_COUNT_DEFAULT
static String	TEZ_APPLICATION_MASTER_CLASS
static String	TEZ_APPLICATION_TYPE
static String	TEZ_CONTAINER_ERR_FILE_NAME
static String	TEZ_CONTAINER_LOG_FILE_NAME
static String	TEZ_CONTAINER_LOG4J_PROPERTIES_FILE
static String	TEZ_CONTAINER_LOGGER_NAME
static String	TEZ_CONTAINER_OUT_FILE_NAME
static String	TEZ_GENERATE_DAG_VIZ
static boolean	TEZ_GENERATE_DAG_VIZ_DEFAULT
static String	TEZ_LIB_URIS The location of the Tez libraries which will be localized for DAGs
static String	TEZ_LOCAL_MODE Tez Local Mode flag.
static boolean	TEZ_LOCAL_MODE_DEFAULT Tez Local Mode flag.
static String	TEZ_PB_BINARY_CONF_NAME
static String	TEZ_PB_PLAN_BINARY_NAME
static String	TEZ_PB_PLAN_TEXT_NAME
static String	TEZ_PREFIX
static String	TEZ_PREWARM_DAG_NAME_PREFIX
static String	TEZ_PROFILE_CONTAINER_LIST Comma separated list of containers which should be profiled.
static String	TEZ_PROFILE_JVM_OPTS The string to be added to the JVM command line for containers being profiled.
static String	TEZ_QUEUE_NAME The queue name for all jobs being submitted as part of a session, or for non session jobs.
static String	TEZ_ROOT_LOGGER_NAME
static String	TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS Time (in seconds) for which the Tez AM should wait for a DAG to be submitted before shutting down.
static int	TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS_DEFAULT
static String	TEZ_SESSION_CLIENT_TIMEOUT_SECS Time (in seconds) to wait for AM to come up when trying to submit a DAG from the client.
static int	TEZ_SESSION_CLIENT_TIMEOUT_SECS_DEFAULT
static String	TEZ_SESSION_LOCAL_RESOURCES_PB_FILE_NAME
static String	TEZ_SESSION_PREFIX Session-related properties
static String	TEZ_SHUFFLE_HANDLER_SERVICE_ID The service id for the NodeManager plugin used to share intermediate data between vertices.
static String	TEZ_SITE_XML
static String	TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS
static int	TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS_DEFAULT
static String	TEZ_TASK_GENERATE_COUNTERS_PER_IO Whether to generate counters per IO or not.
static boolean	TEZ_TASK_GENERATE_COUNTERS_PER_IO_DEFAULT
static String	TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX The maximum amount of time, in milliseconds, to wait before a task asks an AM for another task.
static int	TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX_DEFAULT
static String	TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT
static int	TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT_DEFAULT
static String	TEZ_TASK_PREFIX
static String	YARN_ATS_ENABLED
static boolean	YARN_ATS_ENABLED_DEFAULT

Constructor Summary

TezConfiguration()

TezConfiguration(org.apache.hadoop.conf.Configuration conf)

Method Summary

Methods inherited from class org.apache.hadoop.conf.Configuration

addDefaultResource, addDeprecation, addDeprecation, addDeprecation, addDeprecation, addResource, addResource, addResource, addResource, addResource, clear, dumpConfiguration, dumpDeprecatedKeys, get, get, getBoolean, getClass, getClass, getClassByName, getClassByNameOrNull, getClasses, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getDouble, getEnum, getFile, getFloat, getInstances, getInt, getInts, getLocalPath, getLong, getLongBytes, getPattern, getPropertySources, getProps, getRange, getRaw, getResource, getSocketAddr, getStringCollection, getStrings, getStrings, getTimeDuration, getTrimmed, getTrimmed, getTrimmedStringCollection, getTrimmedStrings, getTrimmedStrings, getValByRegex, isDeprecated, iterator, main, readFields, reloadConfiguration, set, set, setBoolean, setBooleanIfUnset, setClass, setClassLoader, setDouble, setEnum, setFloat, setIfUnset, setInt, setLong, setPattern, setQuietMode, setSocketAddr, setStrings, setTimeDuration, size, toString, unset, updateConnectAddr, write, writeXml, writeXml

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

TEZ_SITE_XML

public static final String TEZ_SITE_XML

See Also:

Constant Field Values

TEZ_PREFIX

public static final String TEZ_PREFIX

See Also:

Constant Field Values

TEZ_AM_PREFIX

public static final String TEZ_AM_PREFIX

See Also:

Constant Field Values

TEZ_TASK_PREFIX

public static final String TEZ_TASK_PREFIX

See Also:

Constant Field Values

TEZ_AM_STAGING_DIR

public static final String TEZ_AM_STAGING_DIR

The staging dir used while submitting DAGs

See Also:

Constant Field Values

TEZ_AM_STAGING_DIR_DEFAULT

public static final String TEZ_AM_STAGING_DIR_DEFAULT

See Also:

Constant Field Values

TEZ_APPLICATION_MASTER_CLASS

public static final String TEZ_APPLICATION_MASTER_CLASS

See Also:

Constant Field Values

TEZ_AM_LOG_LEVEL

public static final String TEZ_AM_LOG_LEVEL

Root Logging level passed to the Tez app master.

See Also:

Constant Field Values

TEZ_AM_LOG_LEVEL_DEFAULT

public static final String TEZ_AM_LOG_LEVEL_DEFAULT

See Also:

Constant Field Values

TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS

public static final String TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS

See Also:

Constant Field Values

TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS_DEFAULT

public static final boolean TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS_DEFAULT

See Also:

Constant Field Values

TEZ_AM_JAVA_OPTS

public static final String TEZ_AM_JAVA_OPTS

Java options for the Tez AppMaster process.

See Also:

Constant Field Values

TEZ_AM_JAVA_OPTS_DEFAULT

public static final String TEZ_AM_JAVA_OPTS_DEFAULT

See Also:

Constant Field Values

TEZ_AM_ENV

public static final String TEZ_AM_ENV

User-provided env for the Tez AM. Any env provided in AMConfiguration overrides env defined by this config property Should be specified as a comma-separated of key-value pairs where each pair is defined as KEY=VAL

See Also:

Constant Field Values

TEZ_AM_CANCEL_DELEGATION_TOKEN

public static final String TEZ_AM_CANCEL_DELEGATION_TOKEN

See Also:

Constant Field Values

TEZ_AM_CANCEL_DELEGATION_TOKEN_DEFAULT

public static final boolean TEZ_AM_CANCEL_DELEGATION_TOKEN_DEFAULT

See Also:

Constant Field Values

TEZ_AM_TASK_LISTENER_THREAD_COUNT

public static final String TEZ_AM_TASK_LISTENER_THREAD_COUNT

See Also:

Constant Field Values

TEZ_AM_TASK_LISTENER_THREAD_COUNT_DEFAULT

public static final int TEZ_AM_TASK_LISTENER_THREAD_COUNT_DEFAULT

See Also:

Constant Field Values

TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL

public static final String TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL

See Also:

Constant Field Values

TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_CLIENT

public static final String TEZ_AM_SECURITY_SERVICE_AUTHORIZATION_CLIENT

See Also:

Constant Field Values

TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT

public static final String TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT

Upper limit on the number of threads user to launch containers in the app master. Expect level config, you shouldn't be needing it in most cases.

See Also:

Constant Field Values

TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT_DEFAULT

public static final int TEZ_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT_DEFAULT

See Also:

Constant Field Values

TEZ_AM_MAX_TASK_FAILURES_PER_NODE

public static final String TEZ_AM_MAX_TASK_FAILURES_PER_NODE

See Also:

Constant Field Values

TEZ_AM_MAX_TASK_FAILURES_PER_NODE_DEFAULT

public static final int TEZ_AM_MAX_TASK_FAILURES_PER_NODE_DEFAULT

See Also:

Constant Field Values

TEZ_AM_MAX_APP_ATTEMPTS

public static final String TEZ_AM_MAX_APP_ATTEMPTS

See Also:

Constant Field Values

TEZ_AM_MAX_APP_ATTEMPTS_DEFAULT

public static int TEZ_AM_MAX_APP_ATTEMPTS_DEFAULT

TEZ_AM_MAX_TASK_ATTEMPTS

public static final String TEZ_AM_MAX_TASK_ATTEMPTS

See Also:

Constant Field Values

TEZ_AM_MAX_TASK_ATTEMPTS_DEFAULT

public static final int TEZ_AM_MAX_TASK_ATTEMPTS_DEFAULT

See Also:

Constant Field Values

TEZ_AM_NODE_BLACKLISTING_ENABLED

public static final String TEZ_AM_NODE_BLACKLISTING_ENABLED

See Also:

Constant Field Values

TEZ_AM_NODE_BLACKLISTING_ENABLED_DEFAULT

public static final boolean TEZ_AM_NODE_BLACKLISTING_ENABLED_DEFAULT

See Also:

Constant Field Values

TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD

public static final String TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD

See Also:

Constant Field Values

TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD_DEFAULT

public static final int TEZ_AM_NODE_BLACKLISTING_IGNORE_THRESHOLD_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CLIENT_THREAD_COUNT

public static final String TEZ_AM_CLIENT_THREAD_COUNT

Number of threads to handle job client RPC requests.

See Also:

Constant Field Values

TEZ_AM_CLIENT_THREAD_COUNT_DEFAULT

public static final int TEZ_AM_CLIENT_THREAD_COUNT_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CLIENT_AM_PORT_RANGE

public static final String TEZ_AM_CLIENT_AM_PORT_RANGE

Range of ports that the AM can use when binding. Leave blank if you want all possible ports.

See Also:

Constant Field Values

TEZ_AM_RESOURCE_MEMORY_MB

public static final String TEZ_AM_RESOURCE_MEMORY_MB

The amount of memory to be used by the AppMaster

See Also:

Constant Field Values

TEZ_AM_RESOURCE_MEMORY_MB_DEFAULT

public static final int TEZ_AM_RESOURCE_MEMORY_MB_DEFAULT

See Also:

Constant Field Values

TEZ_AM_RESOURCE_CPU_VCORES

public static final String TEZ_AM_RESOURCE_CPU_VCORES

See Also:

Constant Field Values

TEZ_AM_RESOURCE_CPU_VCORES_DEFAULT

public static final int TEZ_AM_RESOURCE_CPU_VCORES_DEFAULT

See Also:

Constant Field Values

TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION

public static final String TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION

See Also:

Constant Field Values

TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION_DEFAULT

public static final float TEZ_AM_SLOWSTART_DAG_SCHEDULER_MIN_SHUFFLE_RESOURCE_FRACTION_DEFAULT

See Also:

Constant Field Values

TEZ_AM_PLAN_REMOTE_PATH

public static final String TEZ_AM_PLAN_REMOTE_PATH

The complete path to the serialized dag plan file TEZ_AM_PLAN_PB_BINARY. Used to make the plan available to individual tasks if needed. This will typically be a path in the job submit directory.

See Also:

Constant Field Values

TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX

public static final String TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX

The maximum heartbeat interval between the AM and RM in milliseconds

See Also:

Constant Field Values

TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX_DEFAULT

public static final int TEZ_AM_RM_HEARTBEAT_INTERVAL_MS_MAX_DEFAULT

See Also:

Constant Field Values

TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX

public static final String TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX

The maximum amount of time, in milliseconds, to wait before a task asks an AM for another task.

See Also:

Constant Field Values

TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX_DEFAULT

public static final int TEZ_TASK_GET_TASK_SLEEP_INTERVAL_MS_MAX_DEFAULT

See Also:

Constant Field Values

TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS

public static final String TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS

See Also:

Constant Field Values

TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS_DEFAULT

public static final int TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS_DEFAULT

See Also:

Constant Field Values

TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT

public static final String TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT

See Also:

Constant Field Values

TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT_DEFAULT

public static final int TEZ_TASK_MAX_EVENTS_PER_HEARTBEAT_DEFAULT

See Also:

Constant Field Values

TEZ_TASK_GENERATE_COUNTERS_PER_IO

@InterfaceStability.Unstable
@InterfaceAudience.Private
public static final String TEZ_TASK_GENERATE_COUNTERS_PER_IO

Whether to generate counters per IO or not. Enabling this will rename CounterGroups / CounterNames to making thme unique per Vertex + Src|Destination

See Also:

Constant Field Values

TEZ_TASK_GENERATE_COUNTERS_PER_IO_DEFAULT

public static final boolean TEZ_TASK_GENERATE_COUNTERS_PER_IO_DEFAULT

See Also:

Constant Field Values

TASK_TIMEOUT

public static final String TASK_TIMEOUT

See Also:

Constant Field Values

TASK_HEARTBEAT_TIMEOUT_MS

public static final String TASK_HEARTBEAT_TIMEOUT_MS

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_ENABLED

public static final String TEZ_AM_CONTAINER_REUSE_ENABLED

Configuration to specify whether container should be reused.

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_ENABLED_DEFAULT

public static final boolean TEZ_AM_CONTAINER_REUSE_ENABLED_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED

public static final String TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED

Whether to reuse containers for rack local tasks. Active only if reuse is enabled.

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED_DEFAULT

public static final boolean TEZ_AM_CONTAINER_REUSE_RACK_FALLBACK_ENABLED_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED

public static final String TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED

Whether to reuse containers for non-local tasks. Active only if reuse is enabled.

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED_DEFAULT

public static final boolean TEZ_AM_CONTAINER_REUSE_NON_LOCAL_FALLBACK_ENABLED_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS

public static final String TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS

The amount of time to wait before assigning a container to the next level of locality. NODE - RACK - NON_LOCAL

See Also:

Constant Field Values

TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS_DEFAULT

public static final long TEZ_AM_CONTAINER_REUSE_LOCALITY_DELAY_ALLOCATION_MILLIS_DEFAULT

See Also:

Constant Field Values

TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS

public static final String TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS

The amount of time to hold on to a container if no task can be assigned to it immediately. Only active when reuse is enabled. Set to -1 to never release a container in a session.

See Also:

Constant Field Values

TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS_DEFAULT

public static final long TEZ_AM_CONTAINER_SESSION_DELAY_ALLOCATION_MILLIS_DEFAULT

See Also:

Constant Field Values

TEZ_PB_BINARY_CONF_NAME

public static final String TEZ_PB_BINARY_CONF_NAME

See Also:

Constant Field Values

TEZ_PB_PLAN_BINARY_NAME

public static final String TEZ_PB_PLAN_BINARY_NAME

See Also:

Constant Field Values

TEZ_PB_PLAN_TEXT_NAME

public static final String TEZ_PB_PLAN_TEXT_NAME

See Also:

Constant Field Values

TEZ_CONTAINER_LOG4J_PROPERTIES_FILE

public static final String TEZ_CONTAINER_LOG4J_PROPERTIES_FILE

See Also:

Constant Field Values

TEZ_CONTAINER_LOGGER_NAME

public static final String TEZ_CONTAINER_LOGGER_NAME

See Also:

Constant Field Values

TEZ_ROOT_LOGGER_NAME

public static final String TEZ_ROOT_LOGGER_NAME

See Also:

Constant Field Values

TEZ_CONTAINER_LOG_FILE_NAME

public static final String TEZ_CONTAINER_LOG_FILE_NAME

See Also:

Constant Field Values

TEZ_CONTAINER_ERR_FILE_NAME

public static final String TEZ_CONTAINER_ERR_FILE_NAME

See Also:

Constant Field Values

TEZ_CONTAINER_OUT_FILE_NAME

public static final String TEZ_CONTAINER_OUT_FILE_NAME

See Also:

Constant Field Values

TEZ_LIB_URIS

public static final String TEZ_LIB_URIS

The location of the Tez libraries which will be localized for DAGs

See Also:

Constant Field Values

TEZ_APPLICATION_TYPE

public static final String TEZ_APPLICATION_TYPE

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_COUNT

public static final String TEZ_AM_GROUPING_SPLIT_COUNT

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_BY_LENGTH

public static final String TEZ_AM_GROUPING_SPLIT_BY_LENGTH

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_BY_LENGTH_DEFAULT

public static final boolean TEZ_AM_GROUPING_SPLIT_BY_LENGTH_DEFAULT

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_BY_COUNT

public static final String TEZ_AM_GROUPING_SPLIT_BY_COUNT

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_BY_COUNT_DEFAULT

public static final boolean TEZ_AM_GROUPING_SPLIT_BY_COUNT_DEFAULT

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_WAVES

public static final String TEZ_AM_GROUPING_SPLIT_WAVES

The multiplier for available queue capacity when determining number of tasks for a Vertex. 1.7 with 100% queue available implies generating a number of tasks roughly equal to 170% of the available containers on the queue

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_WAVES_DEFAULT

public static float TEZ_AM_GROUPING_SPLIT_WAVES_DEFAULT

TEZ_AM_GROUPING_SPLIT_MAX_SIZE

public static final String TEZ_AM_GROUPING_SPLIT_MAX_SIZE

Upper bound on the size (in bytes) of a grouped split, to avoid generating excessively large splits.

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_MAX_SIZE_DEFAULT

public static long TEZ_AM_GROUPING_SPLIT_MAX_SIZE_DEFAULT

TEZ_AM_GROUPING_SPLIT_MIN_SIZE

public static final String TEZ_AM_GROUPING_SPLIT_MIN_SIZE

Lower bound on the size (in bytes) of a grouped split, to avoid generating too many splits.

See Also:

Constant Field Values

TEZ_AM_GROUPING_SPLIT_MIN_SIZE_DEFAULT

public static long TEZ_AM_GROUPING_SPLIT_MIN_SIZE_DEFAULT

TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION

public static final String TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION

See Also:

Constant Field Values

TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION_DEFAULT

public static final float TEZ_AM_GROUPING_RACK_SPLIT_SIZE_REDUCTION_DEFAULT

See Also:

Constant Field Values

TEZ_SESSION_PREFIX

public static final String TEZ_SESSION_PREFIX

Session-related properties

See Also:

Constant Field Values

TEZ_SESSION_LOCAL_RESOURCES_PB_FILE_NAME

public static final String TEZ_SESSION_LOCAL_RESOURCES_PB_FILE_NAME

See Also:

Constant Field Values

TEZ_SESSION_CLIENT_TIMEOUT_SECS

public static final String TEZ_SESSION_CLIENT_TIMEOUT_SECS

Time (in seconds) to wait for AM to come up when trying to submit a DAG from the client.

See Also:

Constant Field Values

TEZ_SESSION_CLIENT_TIMEOUT_SECS_DEFAULT

public static final int TEZ_SESSION_CLIENT_TIMEOUT_SECS_DEFAULT

See Also:

Constant Field Values

TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS

public static final String TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS

Time (in seconds) for which the Tez AM should wait for a DAG to be submitted before shutting down.

See Also:

Constant Field Values

TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS_DEFAULT

public static final int TEZ_SESSION_AM_DAG_SUBMIT_TIMEOUT_SECS_DEFAULT

See Also:

Constant Field Values

TEZ_QUEUE_NAME

public static final String TEZ_QUEUE_NAME

The queue name for all jobs being submitted as part of a session, or for non session jobs.

See Also:

Constant Field Values

TEZ_GENERATE_DAG_VIZ

public static final String TEZ_GENERATE_DAG_VIZ

See Also:

Constant Field Values

TEZ_GENERATE_DAG_VIZ_DEFAULT

public static final boolean TEZ_GENERATE_DAG_VIZ_DEFAULT

See Also:

Constant Field Values

TEZ_PROFILE_CONTAINER_LIST

public static final String TEZ_PROFILE_CONTAINER_LIST

Comma separated list of containers which should be profiled.

See Also:

Constant Field Values

TEZ_PROFILE_JVM_OPTS

public static final String TEZ_PROFILE_JVM_OPTS

The string to be added to the JVM command line for containers being profiled.

See Also:

Constant Field Values

TEZ_SHUFFLE_HANDLER_SERVICE_ID

@InterfaceAudience.Private
public static final String TEZ_SHUFFLE_HANDLER_SERVICE_ID

The service id for the NodeManager plugin used to share intermediate data between vertices.

See Also:

Constant Field Values

TEZ_PREWARM_DAG_NAME_PREFIX

@InterfaceAudience.Private
public static final String TEZ_PREWARM_DAG_NAME_PREFIX

See Also:

Constant Field Values

YARN_ATS_ENABLED

public static final String YARN_ATS_ENABLED

See Also:

Constant Field Values

YARN_ATS_ENABLED_DEFAULT

public static final boolean YARN_ATS_ENABLED_DEFAULT

See Also:

Constant Field Values

DAG_RECOVERY_ENABLED

public static final String DAG_RECOVERY_ENABLED

See Also:

Constant Field Values

DAG_RECOVERY_ENABLED_DEFAULT

public static final boolean DAG_RECOVERY_ENABLED_DEFAULT

See Also:

Constant Field Values

DAG_RECOVERY_FILE_IO_BUFFER_SIZE

public static final String DAG_RECOVERY_FILE_IO_BUFFER_SIZE

See Also:

Constant Field Values

DAG_RECOVERY_FILE_IO_BUFFER_SIZE_DEFAULT

public static final int DAG_RECOVERY_FILE_IO_BUFFER_SIZE_DEFAULT

See Also:

Constant Field Values

DAG_RECOVERY_MAX_UNFLUSHED_EVENTS

public static final String DAG_RECOVERY_MAX_UNFLUSHED_EVENTS

See Also:

Constant Field Values

DAG_RECOVERY_MAX_UNFLUSHED_EVENTS_DEFAULT

public static final int DAG_RECOVERY_MAX_UNFLUSHED_EVENTS_DEFAULT

See Also:

Constant Field Values

DAG_RECOVERY_FLUSH_INTERVAL_SECS

public static final String DAG_RECOVERY_FLUSH_INTERVAL_SECS

See Also:

Constant Field Values

DAG_RECOVERY_FLUSH_INTERVAL_SECS_DEFAULT

public static final int DAG_RECOVERY_FLUSH_INTERVAL_SECS_DEFAULT

See Also:

Constant Field Values

DAG_RECOVERY_DATA_DIR_NAME

public static final String DAG_RECOVERY_DATA_DIR_NAME

See Also:

Constant Field Values

DAG_RECOVERY_SUMMARY_FILE_SUFFIX

public static final String DAG_RECOVERY_SUMMARY_FILE_SUFFIX

See Also:

Constant Field Values

DAG_RECOVERY_RECOVER_FILE_SUFFIX

public static final String DAG_RECOVERY_RECOVER_FILE_SUFFIX

See Also:

Constant Field Values

TEZ_LOCAL_MODE

public static final String TEZ_LOCAL_MODE

Tez Local Mode flag. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

TEZ_LOCAL_MODE_DEFAULT

public static final boolean TEZ_LOCAL_MODE_DEFAULT

Tez Local Mode flag. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

TEZ_AM_INLINE_TASK_EXECUTION_ENABLED

public static final String TEZ_AM_INLINE_TASK_EXECUTION_ENABLED

Tez AM Inline Mode flag. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

TEZ_AM_INLINE_TASK_EXECUTION_ENABLED_DEFAULT

public static final boolean TEZ_AM_INLINE_TASK_EXECUTION_ENABLED_DEFAULT

Tez AM Inline Mode flag. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS

public static final String TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS

The maximium number of tasks running in parallel in inline mode. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS_DEFAULT

public static final int TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS_DEFAULT

The maximium number of tasks running in parallel in inline mode. Not valid till Tez-684 get checked-in

See Also:

Constant Field Values

Constructor Detail

TezConfiguration

public TezConfiguration()

TezConfiguration

public TezConfiguration(org.apache.hadoop.conf.Configuration conf)