org.apache.spark.scheduler

Interface MapStatus

public interface MapStatus

Result returned by a ShuffleMapTask to a scheduler. Includes the block manager address that the task has shuffle files stored on as well as the sizes of outputs for each reducer, for passing on to the reduce tasks.

Method Summary

Modifier and Type	Method and Description
long	getSizeForBlock(int reduceId) Estimated size for the reduce block, in bytes.
BlockManagerId	location() Location where this task output is.
long	mapId() The unique ID of this shuffle map task, if spark.shuffle.useOldFetchProtocol enabled we use partitionId of the task or taskContext.taskAttemptId is used.
void	updateLocation(BlockManagerId newLoc)

Method Detail

location

BlockManagerId location()

Location where this task output is.

updateLocation

void updateLocation(BlockManagerId newLoc)

getSizeForBlock

long getSizeForBlock(int reduceId)

Estimated size for the reduce block, in bytes.

If a block is non-empty, then this method MUST return a non-zero size. This invariant is necessary for correctness, since block fetchers are allowed to skip zero-size blocks.

Parameters:

reduceId - (undocumented)

Returns:

(undocumented)

mapId

long mapId()

The unique ID of this shuffle map task, if spark.shuffle.useOldFetchProtocol enabled we use partitionId of the task or taskContext.taskAttemptId is used.

Returns:

(undocumented)