org.apache.twill.yarn

Class YarnTwillRunnerService

java.lang.Object

org.apache.twill.yarn.YarnTwillRunnerService

All Implemented Interfaces:

TwillRunner, TwillRunnerService

public final class YarnTwillRunnerService
extends Object
implements TwillRunnerService

An implementation of TwillRunnerService that runs application on a YARN cluster.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.twill.api.TwillRunner

TwillRunner.LiveInfo

Constructor Summary

Constructors

Constructor and Description
YarnTwillRunnerService(org.apache.hadoop.yarn.conf.YarnConfiguration config, String zkConnect) Creates an instance with a FileContextLocationFactory created base on the given configuration with the user home directory as the location factory namespace.
YarnTwillRunnerService(org.apache.hadoop.yarn.conf.YarnConfiguration config, String zkConnect, LocationFactory locationFactory) Creates an instance.

Method Summary

Methods

Modifier and Type	Method and Description
String	getJVMOptions() Returns any extra JVM options that have been set.
Iterable<TwillController>	lookup(String applicationName) Gets an Iterable of TwillController for all running instances of the given application.
TwillController	lookup(String applicationName, RunId runId) Gets a TwillController for the given application and runId.
Iterable<TwillRunner.LiveInfo>	lookupLive() Gets an Iterable of TwillRunner.LiveInfo.
TwillPreparer	prepare(TwillApplication application) Prepares to run the given TwillApplication as specified by the application.
TwillPreparer	prepare(TwillRunnable runnable) Prepares to run the given TwillRunnable with ResourceSpecification.BASIC resource specification.
TwillPreparer	prepare(TwillRunnable runnable, ResourceSpecification resourceSpecification) Prepares to run the given TwillRunnable with the given resource specification.
Cancellable	scheduleSecureStoreUpdate(SecureStoreUpdater updater, long initialDelay, long delay, TimeUnit unit) Schedules a periodic update of SecureStore.
void	setJVMOptions(String options) This methods sets the extra JVM options that will be passed to the java command line for every application started through this YarnTwillRunnerService instance.
Cancellable	setSecureStoreRenewer(SecureStoreRenewer renewer, long initialDelay, long delay, long retryDelay, TimeUnit unit) Sets and schedules a periodic renewal of SecureStore using a given SecureStoreRenewer.
void	start() Starts the service.
void	stop() Stops the service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnTwillRunnerService

public YarnTwillRunnerService(org.apache.hadoop.yarn.conf.YarnConfiguration config,
                      String zkConnect)

Creates an instance with a FileContextLocationFactory created base on the given configuration with the user home directory as the location factory namespace.

Parameters:

config - Configuration of the yarn cluster

zkConnect - ZooKeeper connection string

YarnTwillRunnerService

public YarnTwillRunnerService(org.apache.hadoop.yarn.conf.YarnConfiguration config,
                      String zkConnect,
                      LocationFactory locationFactory)

Creates an instance.

Parameters:

config - Configuration of the yarn cluster

zkConnect - ZooKeeper connection string

locationFactory - Factory to create Location instances that are readable and writable by this service

Method Detail

start

public void start()

Description copied from interface: TwillRunnerService

Starts the service. Calling this method on a already started instance has no effect. A service that is stopped cannot be started again.

Specified by:

start in interface TwillRunnerService

stop

public void stop()

Description copied from interface: TwillRunnerService

Stops the service. Calling this method on a already stopped instance has no effect.

Specified by:

stop in interface TwillRunnerService

setJVMOptions

public void setJVMOptions(String options)

This methods sets the extra JVM options that will be passed to the java command line for every application started through this YarnTwillRunnerService instance. It only affects applications that are started after options is set. This is intended for advance usage. All options will be passed unchanged to the java command line. Invalid options could cause application not able to start.

Parameters:

options - extra JVM options.

getJVMOptions

public String getJVMOptions()

Returns any extra JVM options that have been set.

See Also:

setJVMOptions(String)

scheduleSecureStoreUpdate

public Cancellable scheduleSecureStoreUpdate(SecureStoreUpdater updater,
                                    long initialDelay,
                                    long delay,
                                    TimeUnit unit)

Description copied from interface: TwillRunner

Schedules a periodic update of SecureStore. The first call to the given SecureStoreUpdater will be made after initialDelay, and subsequently with the given delay between completion of one update and starting of the next. If exception is thrown on call SecureStoreUpdater.update(String, RunId), the exception will only get logged and won't suppress the next update call.

Specified by:

scheduleSecureStoreUpdate in interface TwillRunner

Parameters:

updater - A SecureStoreUpdater for creating new SecureStore.

initialDelay - Delay before the first call to update method.

delay - Delay between completion of one update call to the next one.

unit - time unit for the initialDelay and delay.

Returns:

A Cancellable for cancelling the scheduled update.

setSecureStoreRenewer

public Cancellable setSecureStoreRenewer(SecureStoreRenewer renewer,
                                long initialDelay,
                                long delay,
                                long retryDelay,
                                TimeUnit unit)

Description copied from interface: TwillRunner

Sets and schedules a periodic renewal of SecureStore using a given SecureStoreRenewer. There is always only one active SecureStoreRenewer. Setting a new renewer will replace the old one and setting up a new schedule.

Specified by:

setSecureStoreRenewer in interface TwillRunner

Parameters:

renewer - a SecureStoreRenewer for renewing SecureStore for all applications.

initialDelay - delay before the first call to renew method.

delay - the delay between successful completion of one renew call to the next one.

retryDelay - the delay before the retrying the renewal if the call to SecureStoreRenewer.renew(String, RunId, SecureStoreWriter) raised exception.

unit - time unit for the initialDelay and period.

prepare

public TwillPreparer prepare(TwillRunnable runnable)

Description copied from interface: TwillRunner

Prepares to run the given TwillRunnable with ResourceSpecification.BASIC resource specification. The name for the runnable will be defaulted to runnable.getClass().getSimpleName()

Specified by:

prepare in interface TwillRunner

Parameters:

runnable - The runnable to run through Twill when TwillPreparer.start() is called.

Returns:

A TwillPreparer for setting up runtime options.

prepare

public TwillPreparer prepare(TwillRunnable runnable,
                    ResourceSpecification resourceSpecification)

Description copied from interface: TwillRunner

Prepares to run the given TwillRunnable with the given resource specification. The name for the runnable will be defaulted to runnable.getClass().getSimpleName()

Specified by:

prepare in interface TwillRunner

Parameters:

runnable - The runnable to run through Twill when TwillPreparer.start() is called.

resourceSpecification - The resource specification for running the runnable.

Returns:

A TwillPreparer for setting up runtime options.

prepare

public TwillPreparer prepare(TwillApplication application)

Description copied from interface: TwillRunner

Prepares to run the given TwillApplication as specified by the application.

Specified by:

prepare in interface TwillRunner

Parameters:

application - The application to run through Twill when TwillPreparer.start() is called.

Returns:

A TwillPreparer for setting up runtime options.

lookup

public TwillController lookup(String applicationName,
                     RunId runId)

Description copied from interface: TwillRunner

Gets a TwillController for the given application and runId.

Specified by:

lookup in interface TwillRunner

Parameters:

applicationName - Name of the application.

runId - The runId of the running application.

Returns:

A TwillController to interact with the application or null if no such runId is found.

lookup

public Iterable<TwillController> lookup(String applicationName)

Description copied from interface: TwillRunner

Gets an Iterable of TwillController for all running instances of the given application.

Specified by:

lookup in interface TwillRunner

Parameters:

applicationName - Name of the application.

Returns:

A live Iterable that gives the latest TwillController set for all running instances of the application when Iterable.iterator() is invoked.

lookupLive

public Iterable<TwillRunner.LiveInfo> lookupLive()

Description copied from interface: TwillRunner

Gets an Iterable of TwillRunner.LiveInfo.

Specified by:

lookupLive in interface TwillRunner

Returns:

A live Iterable that gives the latest information on the set of applications that have running instances when Iterable.iterator()} is invoked.