beehive.util

Class StorageSystem

java.lang.Object

beehive.util.StorageSystem

public class StorageSystem
extends Object

The StorageSystem is responsible for keeping track of all of the nodes in the cluster. Nodes can be added, removed, located, and relocated through the StorageSystem. Each Beehive server maintains a StorageSystem. Each StorageSystem only stores the nodes local to that server. If a server requires a remote node, the StorageSystem can fetch it as it caches the location of each node in the entire system.

The StorageSystem can also update and fetch properties of a node as opposed to the entire node itself; this is achieved using NodeData. Bulk processing and parallel processing are also supported.

Field Summary

Fields

Modifier and Type	Field and Description
ConcurrentHashMap<String,String>	locationTable
HashtableServer	myHashTableServer
int	myId
String	myURL
static HashMap<Byte,Class>	nodeClassMap
static HashMap<Byte,String>	nodeClassNameMap
static HashMap<String,Byte>	nodeClassNametoIDMap
int	numServers
int[]	partitionedGraphHostNodesTable
boolean	recordStats
Vector<String>	relocating
static String	RPC_MECHANISM
ConcurrentHashMap<Integer,String>	serverLocationMapping
ConcurrentHashMap<String,Integer>	serverMapping
ConcurrentHashMap<Integer,HashtableServer>	servers
ConcurrentHashMap<Integer,HashTableThriftHandler>	serversThrift
boolean	startCompute
boolean	writeThroughCache

Constructor Summary

Constructors

Constructor and Description
StorageSystem(Vector<String> hostList, Config configParams) Constructor without reference to local hashtable server object, backwards compatible
StorageSystem(Vector<String> hostList, HashtableServer paramHashtableServer, boolean writeThrough, Config configParams, HashMap<Byte,String> nodeClassNameMap, HashMap<Byte,Class> nodeClassMap) Identical to that of StorageSystem(Vector, HashtableServer, Config, HashMap, HashMap) but with the option to enable caching.
StorageSystem(Vector<String> hostList, HashtableServer paramHashtableServer, Config configParams, HashMap<Byte,String> nodeClassNameMap, HashMap<Byte,Class> nodeClassMap) Constructor with reference to local hashtable server object.

Method Summary

Modifier and Type	Method and Description
void	clearStatContainers() Clears all of the statistics associated with the current StorageSystem
void	enableStats() Turns on stat recording
NodeData	exec(NodeData n) Remotely executes the methods of a node.
Vector<NodeData>	execParallel(Vector<NodeData> paramVector) Remotely executes the methods of several nodes in parallel.
Vector<String>	getAllKeys(String url) Gets all of the keys from a particular server
Vector<Node>	getAllNodes()
ClusterMeasures	getClusterMeasures(String url)
Vector<Node>	getLocalNodes(Vector<String> nodeIds) Gets all of the local nodes whose ids are passed in.
Vector<Node>	getLocalNodes(Vector<String> nodeIds, boolean withCaching) Gets all of the local nodes whose ids are passed in.
Vector<NodeData>	getLocalNodesData(Vector<NodeData> nodeData, Vector<NodeData> remoteVector) Gets specific properties of local nodes.
String	getLocation(String nodeId)
String	getLocationURL(String nodeId) Determines the url of the server storing the the node whose nodeId is given.
Node	getNode(String nodeId)
Node	getNode(String nodeId, boolean withCaching) If withCaching is true and caching is enabled through the configuration, then an attempt to get the node from the local cache will be made.
NodeData	getNodeProperties(NodeData n) Gets specific properties of a node
Vector<NodeData>	getNodePropertiesParallel(Vector<NodeData> nodeDatas) Gets the desired properties of several nodes in parallel.
Vector<Node>	getNodesParallel(Vector<String> nodeIds) Gets several Nodes whose ids are passed in; does so in a parallel fashion.
Vector<Node>	getNodesParallel(Vector<String> nodeIds, boolean withCaching) If withCaching is true and caching is enabled through the configuration, then an attempt to get the nodes whose ids are passed in from the cache will be made.
int	getNumberOfServers()
HashtableServer	getServer(String nodeId)
int	getServerID(String nodeId) Gets the id of the server storing the node.
HashtableServer	getServerThrift(String nodeId)
String	getStats()
String	getStats(boolean includeHashtable)
void	putNode(String nodeId, Node node) Puts a node into the local server or a remote server based on the nodeId
void	putNodesParallel(Vector<Node> nodeList) Puts several nodes into the local server or a remote server in parallel based on their respective nodeIds
boolean	relocate(String nodeId) Deprecated. DO NOT USE Relocates a remote node to the calling server. If the node is already local, then nothing happens.
boolean	relocate(String nodeId, boolean lock) Deprecated. DO NOT USE
void	updateNodeProperties(NodeData n) Remotely updates the properties of a node whose nodeId is stored in n.
void	updateNodePropertiesParallel(Vector<NodeData> nodeList) Updates the properties of several nodes in parallel.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

servers

public ConcurrentHashMap<Integer,HashtableServer> servers

serverMapping

public ConcurrentHashMap<String,Integer> serverMapping

locationTable

public ConcurrentHashMap<String,String> locationTable

serverLocationMapping

public ConcurrentHashMap<Integer,String> serverLocationMapping

relocating

public Vector<String> relocating

myURL

public String myURL

myId

public int myId

myHashTableServer

public HashtableServer myHashTableServer

numServers

public int numServers

recordStats

public boolean recordStats

writeThroughCache

public boolean writeThroughCache

startCompute

public boolean startCompute

partitionedGraphHostNodesTable

public int[] partitionedGraphHostNodesTable

serversThrift

public ConcurrentHashMap<Integer,HashTableThriftHandler> serversThrift

RPC_MECHANISM

public static String RPC_MECHANISM

nodeClassNameMap

public static HashMap<Byte,String> nodeClassNameMap

nodeClassMap

public static HashMap<Byte,Class> nodeClassMap

nodeClassNametoIDMap

public static HashMap<String,Byte> nodeClassNametoIDMap

Constructor Detail

StorageSystem

public StorageSystem(Vector<String> hostList,
                     Config configParams)
              throws RemoteException

Constructor without reference to local hashtable server object, backwards compatible

Parameters:

hostList - The names of all of the servers

configParams - The configuration of this particular StorageSystem

Throws:

RemoteException - Thrown if the StorageSystem could not be started

StorageSystem

public StorageSystem(Vector<String> hostList,
                     HashtableServer paramHashtableServer,
                     Config configParams,
                     HashMap<Byte,String> nodeClassNameMap,
                     HashMap<Byte,Class> nodeClassMap)
              throws RemoteException

Constructor with reference to local hashtable server object.

Parameters:

hostList - The names of all of the servers

paramHashtableServer - A handle to the hashtable server this system will call its own

configParams - The configuration of this particular StorageSystem

nodeClassNameMap - Map containing Classnames of all the Node Inherited classes used in the program

nodeClassMap - Map containing Classes of all the Node Inherited classes used in the program

Throws:

RemoteException - Thrown if the StorageSystem could not be started

StorageSystem

public StorageSystem(Vector<String> hostList,
                     HashtableServer paramHashtableServer,
                     boolean writeThrough,
                     Config configParams,
                     HashMap<Byte,String> nodeClassNameMap,
                     HashMap<Byte,Class> nodeClassMap)
              throws RemoteException

Identical to that of StorageSystem(Vector, HashtableServer, Config, HashMap, HashMap) but with the option to enable caching.

Throws:

RemoteException

See Also:

StorageSystem(Vector, HashtableServer, Config, HashMap, HashMap)

Method Detail

getNumberOfServers

public int getNumberOfServers()

Returns:

The number of servers in the cluster

enableStats

public void enableStats()

Turns on stat recording

getNode

public Node getNode(String nodeId)
             throws RemoteException

Parameters:

nodeId - The id of the node that we want to fetch. Can be a local or remote id.

Returns:

The node whose nodeId is passed in.

Throws:

RemoteException - Thrown if the node could not be fetched.

getNode

public Node getNode(String nodeId,
                    boolean withCaching)
             throws RemoteException

If withCaching is true and caching is enabled through the configuration, then an attempt to get the node from the local cache will be made. Otherwise, the behavior is identical to getNode(String)

Parameters:

nodeId - The id of the node that we want to fetch. Can be a local or remote id.

withCaching - True if an attempt to fetch from cache is desired, false otherwise.

Returns:

The node whose nodeId is passed in.

Throws:

RemoteException - Thrown if the node could not be fetched.

getNodesParallel

public Vector<Node> getNodesParallel(Vector<String> nodeIds)
                              throws RemoteException

Gets several Nodes whose ids are passed in; does so in a parallel fashion.

Parameters:

nodeIds - The ids of the desired nodes. The ids can be a mixture of local and remote ids.

Returns:

All of the nodes associated with the ids passed in.

Throws:

RemoteException - Thrown if caching is enabled through the configuration and the nodes could not be fetched from the cache.

getNodesParallel

public Vector<Node> getNodesParallel(Vector<String> nodeIds,
                                     boolean withCaching)
                              throws RemoteException

If withCaching is true and caching is enabled through the configuration, then an attempt to get the nodes whose ids are passed in from the cache will be made. Otherwise, the behavior is identical to getNodesParallel(Vector)

Parameters:

nodeIds - The ids of the nodes we want to fetch. Can be a mixture of local and remote ids.

withCaching - True if an attempt to fetch from cache is desired, false otherwise.

Returns:

All of the nodes whose ids are passed in.

Throws:

RemoteException - Thrown only if caching is desired and the nodes could not be fetched from cache.

getNodeProperties

public NodeData getNodeProperties(NodeData n)
                           throws RemoteException,
                                  InterruptedException

Gets specific properties of a node

Parameters:

n - A NodeData object containing:

• The nodeId of the node whose properties we want to fetch
• A mapping from desired properties to required/desired parameters

Returns:

A NodeData object containing the nodeId of the node and a mapping of the desired properties to their respective values.

Throws:

RemoteException - Thrown if the properties could not be fetched

InterruptedException - Thrown if the properties could not be fetched

getNodePropertiesParallel

public Vector<NodeData> getNodePropertiesParallel(Vector<NodeData> nodeDatas)
                                           throws RemoteException,
                                                  InterruptedException

Gets the desired properties of several nodes in parallel.

Parameters:

nodeDatas - Each NodeData object will contain

• The nodeId of the node whose properties we want to fetch
• A mapping from desired properties to required/desired parameters

Returns:

Each NodeData object will contain the nodeId of the node and a mapping of the desired properties to their respective values.

Throws:

RemoteException - Thrown if a node's properties could not be fetched.

InterruptedException - Thrown if a node's properties could not be fetched.

exec

public NodeData exec(NodeData n)
              throws RemoteException

Remotely executes the methods of a node. If the node is local then the methods are executed locally.

Parameters:

n - Contains the nodeId of the node whose methods are to be executed. The methods that are to be executed are provided as a mapping of method name to parameters.

Returns:

A NodeData object containing the results of the method calls. The NodeData object represents the results as a mapping from method name to return value.

Throws:

RemoteException

execParallel

public Vector<NodeData> execParallel(Vector<NodeData> paramVector)
                              throws RemoteException

Remotely executes the methods of several nodes in parallel. Any nodes that are local are executed locally.

Parameters:

paramVector - A vector of NodeData object containing the nodeIds of nodes whose methods are to be executed

Returns:

A vector of NodeData objects containing the results of the method calls. Each NodeData object represents the results as a mapping from method name to return value for the node whose nodeId is in the NodeData object.

Throws:

RemoteException

putNode

public void putNode(String nodeId,
                    Node node)
             throws RemoteException

Puts a node into the local server or a remote server based on the nodeId

Parameters:

nodeId - The identification of the node that is to be put into the server

node - The node that is to be put into the server

Throws:

RemoteException - Thrown if the node could not be put into the server

putNodesParallel

public void putNodesParallel(Vector<Node> nodeList)
                      throws RemoteException

Puts several nodes into the local server or a remote server in parallel based on their respective nodeIds

Parameters:

nodeList - The nodes that are to be put into the servers.

Throws:

RemoteException - Thrown if a node could not be put into its server

updateNodeProperties

public void updateNodeProperties(NodeData n)
                          throws RemoteException,
                                 org.apache.thrift.TException

Remotely updates the properties of a node whose nodeId is stored in n.

Parameters:

n - A NodeData containing the following:

• The nodeId of the node whose properties we are updating
• The properties of the nodes that we wish to update and their respective new values
• The timestamp of the call

Throws:

RemoteException - Thrown if the update was unsuccessful and if the server is using RMI

org.apache.thrift.TException - Thrown if the update was unsuccessful and if the server is using Thrift

updateNodePropertiesParallel

public void updateNodePropertiesParallel(Vector<NodeData> nodeList)
                                  throws RemoteException

Updates the properties of several nodes in parallel.

Parameters:

nodeList - A vector of NodeData that contain the following:

• The nodeId of the node whose properties we want to update
• The properties of the nodes that we wish to update and their respective new values
• The timestamp of the call

Throws:

RemoteException - Thrown if the update was unsuccessful and if the server is using RMI

getAllKeys

public Vector<String> getAllKeys(String url)
                          throws RemoteException

Gets all of the keys from a particular server

Parameters:

url - The url of the server whose keys we want to fetch.

Returns:

A vector of all of the keys at the provided server url.

Throws:

RemoteException - Thrown if the keys could not be fetched from the provided server url.

getClusterMeasures

public ClusterMeasures getClusterMeasures(String url)
                                   throws RemoteException

Parameters:

url - The url of the server whose ClusterMeasures we want to obtain

Returns:

The ClusterMeasures of the server at the provided url

Throws:

RemoteException - Thrown if the server could not be contacted

relocate

public boolean relocate(String nodeId)
                 throws RemoteException

Deprecated. DO NOT USE

Relocates a remote node to the calling server. If the node is already local, then nothing happens.

Parameters:

nodeId - The id of the node that we want to relocate

Returns:

true if the node was successfully relocated or is already locally stored, false otherwise.

Throws:

RemoteException - Thrown if the node could not be located.

relocate

public boolean relocate(String nodeId,
                        boolean lock)
                 throws RemoteException

Deprecated. DO NOT USE

Throws:

RemoteException

getLocationURL

public String getLocationURL(String nodeId)

Determines the url of the server storing the the node whose nodeId is given.

Parameters:

nodeId - The nodeId of the node whose location we want to determine

Returns:

The url of the server storing the node

getLocation

public String getLocation(String nodeId)

getStats

public String getStats()

Returns:

The current statistics for the StorageSystem

getStats

public String getStats(boolean includeHashtable)
                throws RemoteException

Returns:

The current statistics for the StorageSystem. If includeHashTable is true, then the statistics for the corresponding HashtableServer is also included, otherwise it is not.

Throws:

RemoteException - Thrown if the hashtable server could not be contacted.

getServerID

public int getServerID(String nodeId)

Gets the id of the server storing the node. The id is determined by the hashing scheme defined in the configuration file.

Parameters:

nodeId - The id of the node

Returns:

The id of the server storing the node.

getServer

public HashtableServer getServer(String nodeId)

Parameters:

nodeId - The id of the node

Returns:

A handle on the server storing the node.

getServerThrift

public HashtableServer getServerThrift(String nodeId)

See Also:

getServer(String)

getLocalNodes

public Vector<Node> getLocalNodes(Vector<String> nodeIds)

Gets all of the local nodes whose ids are passed in. If a remote id is passed in, it is not fetched.

Parameters:

nodeIds - The nodeIds of the local nodes we want to fetch.

Returns:

All of the local nodes whose nodeIds are passed in.

getLocalNodesData

public Vector<NodeData> getLocalNodesData(Vector<NodeData> nodeData,
                                          Vector<NodeData> remoteVector)

Gets specific properties of local nodes. The desired properties and nodeIds are passed in as a Vector of NodeData

Parameters:

nodeData - Each NodeData contains the nodeId of the node whose properties we want to fetch. The desired properties are contained within the same NodeData object.

remoteVector - A pointer to a vector of NodeData. Any remote nodeId from nodeData will be added to this.

Returns:

Each NodeData contains the nodeId of a local node from nodeData and the respective properties and values we wanted to fetch.

getLocalNodes

public Vector<Node> getLocalNodes(Vector<String> nodeIds,
                                  boolean withCaching)
                           throws RemoteException

Gets all of the local nodes whose ids are passed in. If withCaching is true and caching is enabled through the configuration, then an attempt to include all of the locally cached nodes will be made. Otherwise, the behavior is identical to getLocalNodes(Vector)

Parameters:

nodeIds - The nodeIds of the local nodes we want to fetch.

withCaching - True if an attempt to include locally cached nodes is desired, false otherwise.

Returns:

All of the local nodes whose nodeIds are passed in.

Throws:

RemoteException - Thrown if the cached nodes could not be fetched.

getAllNodes

public Vector<Node> getAllNodes()
                         throws RemoteException

Returns:

All of the local nodes at this server

Throws:

RemoteException - Thrown if all of the local nodes could not be fetched.

clearStatContainers

public void clearStatContainers()
                         throws RemoteException

Clears all of the statistics associated with the current StorageSystem

Throws:

RemoteException - Thrown if the hashtable server could not be contacted