Developer API

org.apache.spark.storage

Class StorageLevel

Object

org.apache.spark.storage.StorageLevel

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable

public class StorageLevel
extends Object
implements java.io.Externalizable

Flags for controlling the storage of an RDD. Each StorageLevel records whether to use memory, or ExternalBlockStore, whether to drop the RDD to disk if it falls out of memory or ExternalBlockStore, whether to keep the data in memory in a serialized format, and whether to replicate the RDD partitions on multiple nodes.

The StorageLevel$ singleton object contains some static constants for commonly useful storage levels. To create your own storage level object, use the factory method of the singleton object (StorageLevel(...)).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StorageLevel()

Method Summary

Methods

Modifier and Type	Method and Description
static StorageLevel	Developer APIapply(boolean useDisk, boolean useMemory, boolean useOffHeap, boolean deserialized, int replication) Create a new StorageLevel object.
static StorageLevel	Developer APIapply(boolean useDisk, boolean useMemory, boolean deserialized, int replication) Create a new StorageLevel object without setting useOffHeap.
static StorageLevel	Developer APIapply(int flags, int replication) Create a new StorageLevel object from its integer representation.
static StorageLevel	Developer APIapply(java.io.ObjectInput in) Read StorageLevel object from ObjectInput stream.
StorageLevel	clone()
String	description()
boolean	deserialized()
static StorageLevel	DISK_ONLY_2()
static StorageLevel	DISK_ONLY()
boolean	equals(Object other)
static StorageLevel	Developer APIfromString(String s) Return the StorageLevel object with the specified name.
int	hashCode()
boolean	isValid()
static StorageLevel	MEMORY_AND_DISK_2()
static StorageLevel	MEMORY_AND_DISK_SER_2()
static StorageLevel	MEMORY_AND_DISK_SER()
static StorageLevel	MEMORY_AND_DISK()
static StorageLevel	MEMORY_ONLY_2()
static StorageLevel	MEMORY_ONLY_SER_2()
static StorageLevel	MEMORY_ONLY_SER()
static StorageLevel	MEMORY_ONLY()
static StorageLevel	NONE()
static StorageLevel	OFF_HEAP()
void	readExternal(java.io.ObjectInput in)
int	replication()
int	toInt()
String	toString()
boolean	useDisk()
boolean	useMemory()
boolean	useOffHeap()
void	writeExternal(java.io.ObjectOutput out)

Methods inherited from class Object

getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

StorageLevel

public StorageLevel()

Method Detail

NONE

public static StorageLevel NONE()

DISK_ONLY

public static StorageLevel DISK_ONLY()

DISK_ONLY_2

public static StorageLevel DISK_ONLY_2()

MEMORY_ONLY

public static StorageLevel MEMORY_ONLY()

MEMORY_ONLY_2

public static StorageLevel MEMORY_ONLY_2()

MEMORY_ONLY_SER

public static StorageLevel MEMORY_ONLY_SER()

MEMORY_ONLY_SER_2

public static StorageLevel MEMORY_ONLY_SER_2()

MEMORY_AND_DISK

public static StorageLevel MEMORY_AND_DISK()

MEMORY_AND_DISK_2

public static StorageLevel MEMORY_AND_DISK_2()

MEMORY_AND_DISK_SER

public static StorageLevel MEMORY_AND_DISK_SER()

MEMORY_AND_DISK_SER_2

public static StorageLevel MEMORY_AND_DISK_SER_2()

OFF_HEAP

public static StorageLevel OFF_HEAP()

Developer API

fromString

public static StorageLevel fromString(String s)

Return the StorageLevel object with the specified name.

Parameters:

s - (undocumented)

Returns:

(undocumented)

Developer API

apply

public static StorageLevel apply(boolean useDisk,
                 boolean useMemory,
                 boolean useOffHeap,
                 boolean deserialized,
                 int replication)

Create a new StorageLevel object.

Parameters:

useDisk - (undocumented)

useMemory - (undocumented)

useOffHeap - (undocumented)

deserialized - (undocumented)

replication - (undocumented)

Returns:

(undocumented)

Developer API

apply

public static StorageLevel apply(boolean useDisk,
                 boolean useMemory,
                 boolean deserialized,
                 int replication)

Create a new StorageLevel object without setting useOffHeap.

Parameters:

useDisk - (undocumented)

useMemory - (undocumented)

deserialized - (undocumented)

replication - (undocumented)

Returns:

(undocumented)

Developer API

apply

public static StorageLevel apply(int flags,
                 int replication)

Create a new StorageLevel object from its integer representation.

Parameters:

flags - (undocumented)

replication - (undocumented)

Returns:

(undocumented)

Developer API

apply

public static StorageLevel apply(java.io.ObjectInput in)

Read StorageLevel object from ObjectInput stream.

Parameters:

in - (undocumented)

Returns:

(undocumented)

useDisk

public boolean useDisk()

useMemory

public boolean useMemory()

useOffHeap

public boolean useOffHeap()

deserialized

public boolean deserialized()

replication

public int replication()

clone

public StorageLevel clone()

Overrides:

clone in class Object

equals

public boolean equals(Object other)

Overrides:

equals in class Object

isValid

public boolean isValid()

toInt

public int toInt()

writeExternal

public void writeExternal(java.io.ObjectOutput out)

Specified by:

writeExternal in interface java.io.Externalizable

readExternal

public void readExternal(java.io.ObjectInput in)

Specified by:

readExternal in interface java.io.Externalizable

toString

public String toString()

Overrides:

toString in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

description

public String description()