org.apache.tez.client

Class TezClientUtils

java.lang.Object

org.apache.tez.client.TezClientUtils

@InterfaceAudience.Private
public class TezClientUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
TezClientUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static String	addDefaultsToTaskLaunchCmdOpts(String vOpts, org.apache.hadoop.conf.Configuration conf)
static void	addFileSystemCredentialsFromURIs(Collection<URI> uris, org.apache.hadoop.security.Credentials credentials, org.apache.hadoop.conf.Configuration conf) Populate Credentials for the URI's to access them from their FileSystems
static void	addLog4jSystemProperties(String logLevel, List<String> vargs)
static void	addLogParamsToEnv(Map<String,String> environment, String[] logParams)
static org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext	createApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationId appId, DAG dag, String amName, org.apache.tez.client.AMConfiguration amConfig, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> tezJarResources, org.apache.hadoop.security.Credentials sessionCreds, boolean tezLrsAsArchive, TezApiVersionInfo apiVersionInfo, HistoryACLPolicyManager historyACLPolicyManager) Create an ApplicationSubmissionContext to launch a Tez AM
static org.apache.hadoop.fs.FileSystem	ensureStagingDirExists(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path stagingArea) Verify or create the Staging area directory on the configured Filesystem
static DAGClientAMProtocolBlockingPB	getAMProxy(org.apache.hadoop.conf.Configuration conf, String amHost, int amRpcPort, org.apache.hadoop.yarn.api.records.Token clientToAMToken)
static String	maybeAddDefaultLoggingJavaOpts(String logLevel, String javaOpts)
static String	maybeAddDefaultMemoryJavaOpts(String javaOpts, org.apache.hadoop.yarn.api.records.Resource resource, double maxHeapFactor)
static String[]	parseLogParams(String logConfig)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TezClientUtils

public TezClientUtils()

Method Detail

ensureStagingDirExists

public static org.apache.hadoop.fs.FileSystem ensureStagingDirExists(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.fs.Path stagingArea)
                                                              throws IOException

Verify or create the Staging area directory on the configured Filesystem

Parameters:

stagingArea - Staging area directory path

Returns:

the FileSytem for the staging area directory

Throws:

IOException

addFileSystemCredentialsFromURIs

public static void addFileSystemCredentialsFromURIs(Collection<URI> uris,
                                    org.apache.hadoop.security.Credentials credentials,
                                    org.apache.hadoop.conf.Configuration conf)
                                             throws IOException

Populate Credentials for the URI's to access them from their FileSystems

Parameters:

uris - URIs that need to be accessed

credentials - Credentials object into which to add the credentials

conf - Configuration to access the FileSystem

Throws:

IOException

createApplicationSubmissionContext

@InterfaceAudience.Private
public static org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext createApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                                                           DAG dag,
                                                                                                                           String amName,
                                                                                                                           org.apache.tez.client.AMConfiguration amConfig,
                                                                                                                           Map<String,org.apache.hadoop.yarn.api.records.LocalResource> tezJarResources,
                                                                                                                           org.apache.hadoop.security.Credentials sessionCreds,
                                                                                                                           boolean tezLrsAsArchive,
                                                                                                                           TezApiVersionInfo apiVersionInfo,
                                                                                                                           HistoryACLPolicyManager historyACLPolicyManager)
                                                                                                          throws IOException,
                                                                                                                 org.apache.hadoop.yarn.exceptions.YarnException

Create an ApplicationSubmissionContext to launch a Tez AM

Parameters:

appId - Application Id

dag - DAG to be submitted

amName - Name for the application

amConfig - AM Configuration

tezJarResources - Resources to be used by the AM

sessionCreds - the credential object which will be populated with session specific

historyACLPolicyManager -

Returns:

an ApplicationSubmissionContext to launch a Tez AM

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

maybeAddDefaultLoggingJavaOpts

@InterfaceAudience.Private
public static String maybeAddDefaultLoggingJavaOpts(String logLevel,
                                                              String javaOpts)

addDefaultsToTaskLaunchCmdOpts

@InterfaceAudience.Private
public static String addDefaultsToTaskLaunchCmdOpts(String vOpts,
                                                              org.apache.hadoop.conf.Configuration conf)

addLog4jSystemProperties

@InterfaceAudience.Private
public static void addLog4jSystemProperties(String logLevel,
                                                      List<String> vargs)

getAMProxy

@InterfaceAudience.Private
public static DAGClientAMProtocolBlockingPB getAMProxy(org.apache.hadoop.conf.Configuration conf,
                                                                 String amHost,
                                                                 int amRpcPort,
                                                                 org.apache.hadoop.yarn.api.records.Token clientToAMToken)
                                                throws IOException

Throws:

IOException

maybeAddDefaultMemoryJavaOpts

@InterfaceAudience.Private
public static String maybeAddDefaultMemoryJavaOpts(String javaOpts,
                                                             org.apache.hadoop.yarn.api.records.Resource resource,
                                                             double maxHeapFactor)

addLogParamsToEnv

@InterfaceAudience.Private
public static void addLogParamsToEnv(Map<String,String> environment,
                                               String[] logParams)

parseLogParams

@InterfaceAudience.Private
public static String[] parseLogParams(String logConfig)