beehive.util

Class StorageSystem

java.lang.Object

beehive.util.StorageSystem

All Implemented Interfaces:

CheckpointListener

public class StorageSystem
extends java.lang.Object
implements CheckpointListener

Field Summary

Fields

Modifier and Type	Field and Description
HashtableServer	myHashTableServer
int	myId
java.lang.String	myURL
static java.util.HashMap<java.lang.Byte,java.lang.Class>	nodeClassMap
static java.util.HashMap<java.lang.Byte,java.lang.String>	nodeClassNameMap
static java.util.HashMap<java.lang.String,java.lang.Byte>	nodeClassNametoIDMap
int	numServers
int[]	partitionedGraphHostNodesTable
boolean	recordStats
static java.lang.String	RPC_MECHANISM
java.util.concurrent.ConcurrentHashMap<java.lang.Integer,java.lang.String>	serverLocationMapping
java.util.concurrent.ConcurrentHashMap<java.lang.Integer,HashtableServer>	servers

Constructor Summary

Constructors

Constructor and Description
StorageSystem(java.util.Vector<java.lang.String> hostList, boolean writeThrough, Config configParams, java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap, java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap)
StorageSystem(java.util.Vector<java.lang.String> hostList, Config configParams) Constructor without reference to local hashtable server object, backwards compatible
StorageSystem(java.util.Vector<java.lang.String> hostList, Config configParams, java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap, java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap) Only called by BeehiveClientLoader class
StorageSystem(java.util.Vector<java.lang.String> hostList, HashtableServer paramHashtableServer, boolean writeThrough, Config configParams, java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap, java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap) Constructor for caching case
StorageSystem(java.util.Vector<java.lang.String> hostList, HashtableServer paramHashtableServer, Config configParams, java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap, java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap) Constructor with reference to local hashtable server object

Method Summary

Modifier and Type	Method and Description
boolean	acquireLocks(java.util.SortedSet<LockRequest> requests) Attempts to lock each request for validation and commit under a object-locking mechanism
void	checkpoint()
void	clearStatContainers()
void	commitChanges(java.util.Set<Node> writeSet, java.util.Set<NodeData> writeNDSet, java.util.Map<java.lang.String,java.lang.Long> readSet)
void	enableStats() Turn on stat recording
NodeData	exec(NodeData n) Remotely executes the methods of a node.
java.util.Vector<NodeData>	execParallel(java.util.Vector<NodeData> nodeDataVector)
java.util.Vector<java.lang.String>	getAllKeys(java.lang.String url)
java.util.Vector<Node>	getAllLocalNodes()
java.util.Vector<Node>	getLocalNodes(java.util.Vector<java.lang.String> nodeIds) Gets all of the local nodes whose ids are passed in.
java.util.Vector<Node>	getLocalNodes(java.util.Vector<java.lang.String> nodeIds, boolean withCaching) Gets all of the local nodes whose ids are passed in.
java.util.Vector<NodeData>	getLocalNodesData(java.util.Vector<NodeData> nodeData, java.util.Vector<NodeData> remoteVector) Deprecated.
java.lang.String	getLocation(java.lang.String nodeId)
Node	getNode(java.lang.String nodeId)
Node	getNode(java.lang.String nodeId, boolean withCaching) If withCaching is true and caching is enabled through the configuration, then an attempt to get the node from the local cache will be made.
NodeData	getNodeProperties(NodeData n) Gets specific properties of a node
java.util.Vector<NodeData>	getNodePropertiesParallel(java.util.Vector<NodeData> nodeDatas) Gets the desired properties of several nodes in parallel.
java.util.Vector<Node>	getNodesParallel(java.util.Vector<java.lang.String> nodeIds) Gets several Nodes whose ids are passed in; does so in a parallel fashion.
java.util.Vector<Node>	getNodesParallel(java.util.Vector<java.lang.String> nodeIds, boolean withCaching) If withCaching is true and caching is enabled through the configuration, then an attempt to get the nodes whose ids are passed in from the cache will be made.
int	getNumberOfServers()
HashtableServer	getServer(java.lang.String nodeId)
int	getServerID(java.lang.String nodeId)
java.lang.String	getStats()
java.lang.String	getStats(boolean includeHashtable) External API to get stats
boolean	putNode(java.lang.String nodeId, Node node) Puts a node into the local server or a remote server based on the nodeId
void	putNodesParallel(java.util.Vector<Node> nodeList) Bulk call to putNode, called by the application Calls are bucketised to the respective servers
boolean	relocate(java.lang.String nodeId)
boolean	relocate(java.lang.String nodeId, boolean lock)
boolean	updateNodeProperties(NodeData n) update Reflection API for the storage system
void	updateNodePropertiesParallel(java.util.Vector<NodeData> nodeDataVector) updateNodeProperties for the bulk case

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

servers

public java.util.concurrent.ConcurrentHashMap<java.lang.Integer,HashtableServer> servers

serverLocationMapping

public java.util.concurrent.ConcurrentHashMap<java.lang.Integer,java.lang.String> serverLocationMapping

myURL

public java.lang.String myURL

myId

public int myId

myHashTableServer

public HashtableServer myHashTableServer

numServers

public int numServers

recordStats

public boolean recordStats

partitionedGraphHostNodesTable

public int[] partitionedGraphHostNodesTable

RPC_MECHANISM

public static java.lang.String RPC_MECHANISM

nodeClassNameMap

public static java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap

nodeClassMap

public static java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap

nodeClassNametoIDMap

public static java.util.HashMap<java.lang.String,java.lang.Byte> nodeClassNametoIDMap

Constructor Detail

StorageSystem

public StorageSystem(java.util.Vector<java.lang.String> hostList,
                     Config configParams)
              throws java.rmi.RemoteException

Constructor without reference to local hashtable server object, backwards compatible

Parameters:

hostList -

Throws:

java.rmi.RemoteException

StorageSystem

public StorageSystem(java.util.Vector<java.lang.String> hostList,
                     HashtableServer paramHashtableServer,
                     Config configParams,
                     java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap,
                     java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap)
              throws java.rmi.RemoteException

Constructor with reference to local hashtable server object

Parameters:

hostList -

paramHashtableServer -

configParams -

nodeClassNameMap - : Map containing Classnames of all the Node Inherited classes used in the program

nodeClassMap - : Map containing Classes of all the Node Inherited classes used in the program

Throws:

java.rmi.RemoteException

StorageSystem

public StorageSystem(java.util.Vector<java.lang.String> hostList,
                     Config configParams,
                     java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap,
                     java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap)
              throws java.rmi.RemoteException

Only called by BeehiveClientLoader class

Parameters:

hostList -

configParams -

nodeClassNameMap -

nodeClassMap -

Throws:

java.rmi.RemoteException

StorageSystem

public StorageSystem(java.util.Vector<java.lang.String> hostList,
                     HashtableServer paramHashtableServer,
                     boolean writeThrough,
                     Config configParams,
                     java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap,
                     java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap)
              throws java.rmi.RemoteException

Constructor for caching case

Parameters:

hostList -

paramHashtableServer -

writeThrough -

Throws:

java.rmi.RemoteException

StorageSystem

public StorageSystem(java.util.Vector<java.lang.String> hostList,
                     boolean writeThrough,
                     Config configParams,
                     java.util.HashMap<java.lang.Byte,java.lang.String> nodeClassNameMap,
                     java.util.HashMap<java.lang.Byte,java.lang.Class> nodeClassMap)
              throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

Method Detail

getNumberOfServers

public int getNumberOfServers()

enableStats

public void enableStats()

Turn on stat recording

getNode

public Node getNode(java.lang.String nodeId)
             throws java.rmi.RemoteException

Parameters:

nodeId - The id of the node that we want to fetch. Can be a local or remote id.

Returns:

The node whose nodeId is passed in.

Throws:

java.rmi.RemoteException - Thrown if the node could not be fetched.

getNode

public Node getNode(java.lang.String nodeId,
                    boolean withCaching)
             throws java.rmi.RemoteException

If withCaching is true and caching is enabled through the configuration, then an attempt to get the node from the local cache will be made. Otherwise, the behavior is identical to getNode(String)

Parameters:

nodeId - The id of the node that we want to fetch. Can be a local or remote id.

withCaching - True if an attempt to fetch from cache is desired, false otherwise.

Returns:

The node whose nodeId is passed in.

Throws:

java.rmi.RemoteException - Thrown if the node could not be fetched.

getNodesParallel

public java.util.Vector<Node> getNodesParallel(java.util.Vector<java.lang.String> nodeIds)
                                        throws java.rmi.RemoteException

Gets several Nodes whose ids are passed in; does so in a parallel fashion.

Parameters:

nodeIds - The ids of the desired nodes. The ids can be a mixture of local and remote ids.

Returns:

All of the nodes associated with the ids passed in.

Throws:

java.rmi.RemoteException - Thrown if caching is enabled through the configuration and the nodes could not be fetched from the cache.

getNodesParallel

public java.util.Vector<Node> getNodesParallel(java.util.Vector<java.lang.String> nodeIds,
                                               boolean withCaching)
                                        throws java.rmi.RemoteException

If withCaching is true and caching is enabled through the configuration, then an attempt to get the nodes whose ids are passed in from the cache will be made. Otherwise, the behavior is identical to getNodesParallel(Vector)

Parameters:

nodeIds - The ids of the nodes we want to fetch. Can be a mixture of local and remote ids.

withCaching - True if an attempt to fetch from cache is desired, false otherwise.

Returns:

All of the nodes whose ids are passed in.

Throws:

java.rmi.RemoteException - Thrown only if caching is desired and the nodes could not be fetched from cache.

getNodeProperties

public NodeData getNodeProperties(NodeData n)
                           throws java.rmi.RemoteException,
                                  java.lang.InterruptedException

Gets specific properties of a node

Parameters:

n - A NodeData object containing:

• The nodeId of the node whose properties we want to fetch
• A mapping from desired properties to required/desired parameters

Returns:

A NodeData object containing the nodeId of the node and a mapping of the desired properties to their respective values.

Throws:

java.rmi.RemoteException - Thrown if the properties could not be fetched

java.lang.InterruptedException - Thrown if the properties could not be fetched

getNodePropertiesParallel

public java.util.Vector<NodeData> getNodePropertiesParallel(java.util.Vector<NodeData> nodeDatas)
                                                     throws java.rmi.RemoteException,
                                                            java.lang.InterruptedException

Gets the desired properties of several nodes in parallel.

Parameters:

nodeDatas - Each NodeData object will contain

• The nodeId of the node whose properties we want to fetch
• A mapping from desired properties to required/desired parameters

Returns:

Each NodeData object will contain the nodeId of the node and a mapping of the desired properties to their respective values.

Throws:

java.rmi.RemoteException - Thrown if a node's properties could not be fetched.

java.lang.InterruptedException - Thrown if a node's properties could not be fetched.

exec

public NodeData exec(NodeData n)
              throws java.rmi.RemoteException

Remotely executes the methods of a node. If the node is local then the methods are executed locally.

Parameters:

n - Contains the nodeId of the node whose methods are to be executed. The methods that are to be executed are provided as a mapping of method name to parameters.

Returns:

A NodeData object containing the results of the method calls. The NodeData object represents the results as a mapping from method name to return value.

Throws:

java.rmi.RemoteException

execParallel

public java.util.Vector<NodeData> execParallel(java.util.Vector<NodeData> nodeDataVector)
                                        throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

putNode

public boolean putNode(java.lang.String nodeId,
                       Node node)
                throws java.rmi.RemoteException

Puts a node into the local server or a remote server based on the nodeId

Parameters:

nodeId - The identification of the node that is to be put into the server

node - The node that is to be put into the server

Throws:

java.rmi.RemoteException - Thrown if the node could not be put into the server

putNodesParallel

public void putNodesParallel(java.util.Vector<Node> nodeList)
                      throws java.rmi.RemoteException

Bulk call to putNode, called by the application Calls are bucketised to the respective servers

Parameters:

nodeList - The nodes that are to be put into the servers.

Throws:

java.rmi.RemoteException - Thrown if a node could not be put into its server

updateNodeProperties

public boolean updateNodeProperties(NodeData n)
                             throws java.rmi.RemoteException,
                                    org.apache.thrift.TException

update Reflection API for the storage system

Parameters:

n - - this is a Node data which tells what the updated properties and TS are

Returns:

Returns the same node data but the property map now contains the updates that failed if property map = null, then nothing failed, ALL OK if not null, then it contains exception objects for the failures

Throws:

java.rmi.RemoteException

org.apache.thrift.TException

updateNodePropertiesParallel

public void updateNodePropertiesParallel(java.util.Vector<NodeData> nodeDataVector)
                                  throws java.rmi.RemoteException

updateNodeProperties for the bulk case

Parameters:

nodeDataVector -

Throws:

java.rmi.RemoteException

getAllKeys

public java.util.Vector<java.lang.String> getAllKeys(java.lang.String url)
                                              throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

relocate

public boolean relocate(java.lang.String nodeId)
                 throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

relocate

public boolean relocate(java.lang.String nodeId,
                        boolean lock)
                 throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

getLocation

public java.lang.String getLocation(java.lang.String nodeId)

getStats

public java.lang.String getStats()

getStats

public java.lang.String getStats(boolean includeHashtable)
                          throws java.rmi.RemoteException

External API to get stats

Parameters:

includeHashtable -

Returns:

Throws:

java.rmi.RemoteException

getServerID

public int getServerID(java.lang.String nodeId)

getServer

public HashtableServer getServer(java.lang.String nodeId)

getLocalNodes

public java.util.Vector<Node> getLocalNodes(java.util.Vector<java.lang.String> nodeIds)

Gets all of the local nodes whose ids are passed in. If a remote id is passed in, it is not fetched.

Parameters:

nodeIds - The nodeIds of the local nodes we want to fetch.

Returns:

All of the local nodes whose nodeIds are passed in.

getLocalNodesData

@Deprecated
public java.util.Vector<NodeData> getLocalNodesData(java.util.Vector<NodeData> nodeData,
                                                                java.util.Vector<NodeData> remoteVector)

Deprecated.

Gets specific properties of local nodes. The desired properties and nodeIds are passed in as a Vector of NodeData

Parameters:

nodeData - Each NodeData contains the nodeId of the node whose properties we want to fetch. The desired properties are contained within the same NodeData object.

remoteVector - A pointer to a vector of NodeData. Any remote nodeId from nodeData will be added to this.

Returns:

Each NodeData contains the nodeId of a local node from nodeData and the respective properties and values we wanted to fetch.

getLocalNodes

public java.util.Vector<Node> getLocalNodes(java.util.Vector<java.lang.String> nodeIds,
                                            boolean withCaching)
                                     throws java.rmi.RemoteException

Gets all of the local nodes whose ids are passed in. If withCaching is true and caching is enabled through the configuration, then an attempt to include all of the locally cached nodes will be made. Otherwise, the behavior is identical to getLocalNodes(Vector)

Parameters:

nodeIds - The nodeIds of the local nodes we want to fetch.

withCaching - True if an attempt to include locally cached nodes is desired, false otherwise.

Returns:

All of the local nodes whose nodeIds are passed in.

Throws:

java.rmi.RemoteException - Thrown if the cached nodes could not be fetched.

getAllLocalNodes

public java.util.Vector<Node> getAllLocalNodes()
                                        throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

clearStatContainers

public void clearStatContainers()
                         throws java.rmi.RemoteException

Throws:

java.rmi.RemoteException

acquireLocks

public boolean acquireLocks(java.util.SortedSet<LockRequest> requests)

Attempts to lock each request for validation and commit under a object-locking mechanism

Parameters:

requests - Metadata of nodes that are to be locked

Returns:

true if all lock requests were acquired, false otherwise

commitChanges

public void commitChanges(java.util.Set<Node> writeSet,
                          java.util.Set<NodeData> writeNDSet,
                          java.util.Map<java.lang.String,java.lang.Long> readSet)

checkpoint

public void checkpoint()

Specified by:

checkpoint in interface CheckpointListener