/**

\page data_structure Parallel Data Structures
\tableofcontents

@section data_structure Parallel Data Structures

@subsection graphs Graphs

\image html galois_parallel_graphs.png "Galois Parallel Graphs"

The current available galois parallel graphs can be seen in the above figure where shaded boxes represent conceptual graph class for classifying the graph data structures. Broadly, there are two kinds of Galois parallel graph structures: morph graphs and local computation graphs. Morph graphs allow graph mutations while local computation graphs do not allow any modification to the graph topology. These graphs are described in details in the following sections. In order to provide high performance, in general those graphs do not provide direct access to the incoming edges and neighbors given a node unless such functionality is specified. 

@subsubsection localcompgraph Local Computation Graphs 

   A local computation graph does not allow graph mutation. Details about how to construct a local computation graphs can be seen in \ref graphio.

<hr>
<H4> @link galois::graphs::LC_CSR_Graph </H4>
   
Internally, {@link galois::graphs::LC_CSR_Graph} uses Compressed Sparse Row format data structure to represent the graph.  
 
   - Defining a graph
     
     The {@link galois::graphs::LC_CSR_Graph} takes five template parameters to customize the graph:

     \li NodeTy: the type of data stored on each node. Void can be used when no data is stored on each node.
     
     \li EdgeTy: the type of data stored on each edge. Void can be used when no data is stored on each edge.
         
     \li HasNoLockable: indicates if abstract locks will be enabled in the graph or not. By default, it is true.  

     \li UseNumaAlloc: if true, NUMA-aware graph allocation will be used. By default, it is false.
    
     \li HasOutOfLineLockable: if true, the graph will store abstract locks separate from nodes. By default, it is false. 
     
     HasNoLockable, UseNumaAlloc and HasOutOfLineLockable are optional.  

     An example to define a local computation graph can be seen as follows:

     In {@link apps/sssp/SSSP.cpp}

     @snippet apps/sssp/SSSP.cpp Define LC_CSR_Graph

   - Accessing graph topology
     
     The number of nodes in the graph can be accessed by using {@link galois::graphs::LC_CSR_Graph::size}. The number of edges in the graph can be accessed by using {@link galois::graphs::LC_CSR_Graph::sizeEdges}. 

     The nodes in the graph can be accessed by the node iterator {@link galois::graphs::LC_CSR_Graph::iterator} with {@link galois::graphs::LC_CSR_Graph::begin} and {@link galois::graphs::LC_CSR_Graph::end}. 

     The outgoing edges of a node can be accessed by the edge iterator {@link galois::graphs::LC_CSR_Graph::edge_iterator} with {@link galois::graphs::LC_CSR_Graph::edge_begin} and {@link galois::graphs::LC_CSR_Graph::edge_end}. Then the outgoing neighbors can be accessed by {@link galois::graphs::LC_CSR_Graph::getEdgeDst} given a outgoing edge. 

     An example of accessing the topology of the graph can be seen as follows:

     In {@link apps/tutorial/SSSPsimple.cpp}
    
     @snippet apps/tutorial/SSSPsimple.cpp loop over neighbors

   - Accessing node data and edge data
    
     Given a node or edge, the data on them can be accessed by {@link galois::graphs::LC_CSR_Graph::getData} or {@link galois::graphs::LC_CSR_Graph::getEdgeData} correspondingly. 
    
     An example of accessing node data and edge data can be seen as follows:

     In {@link apps/tutorial/SSSPsimple.cpp}
  
     @snippet apps/tutorial/SSSPsimple.cpp get edge and node data

   - Other Functionalities
     
    {@link galois::graphs::LC_CSR_Graph::sortEdges} and {@link galois::graphs::LC_CSR_Graph::sortEdgesByEdgeData} can be used to make the outgoing edges of a node sorted. The two functions differ by the comparison functions. In {@link galois::graphs::LC_CSR_Graph::sortEdgesByEdgeData}, the comparison function is over edge data type EdgeTy. In {@link galois::graphs::LC_CSR_Graph::sortEdges}, the comparison function is over {@link galois::graphs::EdgeSortValue} which represents a pair of GraphNode and EdgeTy.   
 
    An example of sort edges can be seen as follows:

    In {@link apps/boruvka/Boruvka.cpp}
  
    @snippet apps/boruvka/Boruvka.cpp sortEdgeByEdgeData


<hr>

<H4> @link galois::graphs::LC_Linear_Graph </H4>

A variant of {@link galois::graphs::LC_CSR_Graph} where the difference is the physical layouts of the data structures are slightly different but the functionalities of the two graphs are the same. Depending on the applications, the two graphs may achieve different performance from other variants.

An example to define a {@link galois::graphs::LC_Linear_Graph} can be seen as follows:

In {@link apps/tutorial/SSSPsimple.cpp}

@snippet apps/tutorial/SSSPsimple.cpp Define LC Graph

<hr>
<H4> @link galois::graphs::LC_InlineEdge_Graph </H4>

A variant of {@link galois::graphs::LC_CSR_Graph} where the difference is the physical layouts of the data structure are slightly different but the functionalities of those graphs are the same. Depending on the applications, the graph may achieve different performance from other variants.

An example to define a {@link galois::graphs::LC_InlineEdge_Graph} can be seen as follows:

In {@link apps/sssp/SSSP.cpp}

@snippet apps/sssp/SSSP.cpp Define LC_InlineEdge_Graph

<hr>

<H4> @link galois::graphs::LC_InOut_Graph </H4>

{@link galois::graphs::LC_InOut_Graph} provides accessing incoming neighbors besides the functionalities provided {@link galois::graphs::LC_CSR_Graph}.   
   
   - Accessing incoming edges and neighbors of a node

     The incoming edges of a node can be accessed by {@link galois::graphs::LC_InOut_Graph::in_edge_iterator} with {@link galois::graphs::LC_InOut_Graph::in_edge_begin}. The data associated with the incoming edge can be accessed using {galois::graphs::LC_CSR_Graph::getInEdgeData}. Given an incoming edge, the incoming neighbor node can be accessed using {@link galois::graphs::LC_InOut_Graph::getInEdgeDst}.       

    In {@link apps/pagerank/PageRank.cpp}

    An example to define a {@link galois::graphs::LC_InOut_Graph} can be seen as follows:

    @snippet apps/pagerank/PageRank.cpp Define LC_InOut_Graph

    An example to access the incoming edges and neighbors of a node can be seen as follows:

    @snippet apps/pagerank/PageRank.cpp Access in-neighbors of LC_InOut_Graph

<hr> 


@subsubsection morphgraph Morph Graphs 

Morph graphs allow concurrent graph mutations.

<hr>
<H4> @link galois::graphs::MorphGraph </H4>

   - Defining a graph

     The {@link galois::graphs::MorphGraph} takes four template parameters to customize the graph:

     \li NodeTy: the type of data stored on each node. Void can be used when no data is stored on each edge. 
     
     \li EdgeTy: the type of data stored on each edge. Void can be used when no data is stored on each edge.
     
     \li Directional: if true, the graph is a directional graph; otherwise, the graph is an un-directional graph. By default, it is true. 
     
     \li HasNoLockable: if true, abstract locks will be not enabled. By default, abstract lock is enabled.  

     Directional and HasNoLockable are optional.  

     An example to define a {@link galois::graphs::MorphGraph} can be seen as follows:

     In {@link apps/tutorial/Torus.cpp}

     @snippet apps/tutorial/Torus.cpp define a graph
     
     @snippet apps/delaunayrefinement/Subgraph.h MorphGraph Definition in DMR

   - Mutating a graph
    
     A graph node can be created using {@link galois::graphs::MorphGraph::createNode}. If the node data type specified is not void, {@link galois::graphs::MorphGraph::createNode} will create a node data object internally. The parameters of {@link galois::graphs::MorphGraph::createNode} will passed to the constuctor of the node data type to create the node data instance. There are three cases: 
     
     \li if no parameter is passed into {@link galois::graphs::MorphGraph::createNode}, then the default constructor of the node data type will be called to create the node data instance. 
     \li if some parameters are passed into {@link galois::graphs::MorphGraph::createNode}, the corresponding constructor of the node data type will be called to create the node data instance.
     \li if a node data instance is passed into {@link galois::graphs::MorphGraph::createNode}, then the copy constrcutor of the node data type will be called to create the node data instance.
     
     if the node data type specified is void, then {@link galois::graphs::MorphGraph::createNode} should be called without paramters. {@link galois::graphs::MorphGraph::createNode} will return an instance of {galois::graphs::FistGraph::GraphNode} which stands for a reference to the created node. Through this instance, the created node can be added to the graph using {@link galois::graphs::MorphGraph::addNode}. An existing node in the graph can be removed using {@link galois::graphs::MorphGraph::removeNode}, the parameter of which is also an instace of {galois::graphs::FistGraph::GraphNode}. 
     
     An example of creating nodes in the graph can be seen as follows:
      
     In {@link apps/tutorial/Torus.cpp}
     
     @snippet apps/tutorial/Torus.cpp create and add node

     A graph edge can be created and added into a graph using {@link galois::graphs::MorphGraph::addEdge}.In the directional case, the paramters src and dst of {@link galois::graphs::MorphGraph::addEdge} represents the source node and the destination node of the edge to be added. When the graph is un-directional, any order of the two ending nodes of the edge to be added will be treated as the same edge and can be passed into this function with equal effects. {@link galois::graphs::MorphGraph::addEdge} will first detect whether the edge to be added exists so that the graph does not contain multi-edges. If such edge does not exist, this function will create an edge in the graph with an edge data object initialized to the default value of the edge data type; otherwise, it will not add new edge into the graph. In either case, the function will return an edge_iterator pointing to the added edge. Passing this edge_iterator to {@link galois::graphs::MorphGraph::getEdgeData} will return a reference to the edge data object associated with the added edge. Through the returned reference, the edge data object can be modified. To add multi-edges into the graph, {@link galois::graphs::MorphGraph::addMultiEdge} can be used. This function does not check if the edge to be added exists, so it is more efficient than {@link galois::graphs::MorphGraph::addEdge}. An edge can be removed by using {@link galois::graphs::MorphGraph::removeEdge}.

     An example of creating edges can be seen as follows:
    
     In {@link apps/tutorial/Torus.cpp} 
   
     @snippet apps/tutorial/Torus.cpp add edges
  
     {@link galois::graphs::MorphGraph::resizeEdges} is used to pre-allocate space for edges of a graph node.

   - Accessing graph topology
    
     The number of nodes in the graph can be accessed by using {@link galois::graphs::MorphGraph::size}. Note that this function is not threadsafe. {@link galois::graphs::MorphGraph::sizeOfEdgeData} can be used to return size of the edge data.

     The nodes in the graph can be accessed by the node iterator {@link galois::graphs::MorphGraph::iterator} with {@link galois::graphs::MorphGraph::begin} and {@link galois::graphs::MorphGraph::end}. 

     The outgoing edges of a node can be accessed by the edge iterator {@link galois::graphs::MorphGraph::edge_iterator} with {@link galois::graphs::MorphGraph::edge_begin} and {@link galois::graphs::MorphGraph::edge_end}. Then the outgoing neighbors can be accessed by {@link galois::graphs::MorphGraph::getEdgeDst} given a outgoing edge. This graph does not provide direct access to the incoming edges and neighbors of a node.  

    An example to access the outgoing neighbors of a node can be seen as follows:

    @snippet apps/delaunayrefinement/Cavity.h Accessing neighbors of a node in DMR
    @snippet apps/tutorial/Torus.cpp loop over neighbors

   - Accessing node data and edge data   
 
    Given a node or edge, the data on them can be accessed by {@link galois::graphs::MorphGraph::getData} or {@link galois::graphs::MorphGraph::getEdgeData} correspondingly. 

    An example to access the data of a node can be seen as follows:

    @snippet apps/tutorial/Torus.cpp access node data


<hr> 

<H4> @link galois::graphs::LC_Morph_Graph </H4>

This graph extends {@link galois::graphs::LC_CSR_Graph} to give limited graph mutation ability. When creating a node, the users should specify the maximum allowed degree of the node. The outgoing edges of this node should not exceed this degree. {@link galois::graphs::LC_Morph_Graph::createNode} is used to create node and its first parameter "nedges" is used to specify the maximum allowed degree. The remaining parameters of {@link galois::graphs::LC_Morph_Graph::createNode} will be passed to the constructor of the node data type to create the node data instance, same as the parameters of {@link galois::graphs::MorphGraph::createNode}. Importantly, removing nodes in the graph is not supported. Edges can be removed using {@link galois::graphs::LC_Morph_Graph::removeEdge}. Note removing edges will invalidate the edge_iterator if an edge_iterator is used. Other functionalities of {@link galois::graphs::LC_Morph_Graph} is same as {@link galois::graphs::LC_CSR_Graph} which can be seen in \ref localcompgraph 

<hr> 


@subsubsection methodFlags Method flags for controlling conflict detection
     
When using abstract locks, whenever a graph operation is called in parallel region, the runtime will check if the nodes or edges the graph operations will access has been locked by other threads. If yes, the runtime will roll back the computation. Otherwise, the runtime will log the information for un-do the computation when roll back happens. There are overheads associated with such conflict detection. The users can control such conflict detection by passing method flags into the graph operations to indicate the runtime not performing certain checks. One of {@link galois::MethodFlag::WRITE}, {@link galois::READ_INTENT}, or {@link galois::MethodFlag::UNPROTECTED}, that dictates what support the runtime will provide for each call. By default, the runtime assumes galois::WRITE. Note: the current version of Galois, version 2.1, does not support saving undo information. Version 2.0 does support saving undo information. 

@subsection reduction Reduction Operations

The relationships among the reducers available in Galois can be seen in the above figure. The shaded boxes are not real classes but for classifying the reducers. The reducers fall into two categories, {@link galois::GSimpleReducible} and {@link galois::GBigReducible}. GSimpleReducible are trying to reduce values of primitive types or types with low copy cost; e.g. GAccumulator is for accumulating values such as ints. The details can be seen in the following sections.     

\image html galois_parallel_reducers.png "Galois Parallel Reducers"

@subsubsection greducible General Reducer 

<H4> @link galois::GSimpleReducible </H4>

  - Defining a reducer

{@link galois::GSimpleReducible} is used to reduce mutliple values of type T to a single value. It is suitable for  basic types or low copy overhead types. It takes the type T of the values to be reduced and a functor BinFunc as template parameters, where the BinFunc conforms to:
    
    T operator()(const T& a, const T& b);

  - Reducing values
   
    {@link galois::GSimpleReducible} starts with the default value of Type T. {@link galois::GSimpleReducible::update} is used to provide new values to reduce in parallel. After parallel region, the final value can be retrieved using {@link galois::GSimpleReducible::reduce}  


    
<H4> @link galois::GBigReducible </H4>
    {@link galois::GBigReducible} is used to reduce a set of large or non-trivial objects of type T to a single copy. The underlying assumption is that T is expensive to copy.  The functor BinFunc takes two arguments and applies the reduction in such a way the left argument is updated inplace.   

    The BinFunc functor should conform to:

        void operator()(T& lhs, const T& rhs)

    Here the lhs is updated in place to reflect the output of the reduction operator. 

    An example of defining a BinFunc can be seen in the {@link apps/bfs/Diameter.cpp}:
    
    @snippet apps/bfs/Diameter.cpp Define BinFunc for GReducible 
    
    Given the BinFunc, defining a {@link galois::GReducible} can be seen in the {@link apps/bfs/Diameter.cpp}:

    @snippet apps/bfs/Diameter.cpp Define GReducible 

    An example of using {@link galois::GReducible} in parallel region can be seen in the {@link apps/bfs/Diameter.cpp}:

    @snippet apps/bfs/Diameter.cpp Use GReducible in parallel 

    @snippet apps/bfs/Diameter.cpp Reduce the final value

@subsubsection specialreducers Special Reducers

There are several built-in special reducers.

 
{@link galois::GReduceMax} and {@link galois::GReduceMin} are used to reduce the maximum and minimum values. The type T has to meet the type requirements of std::max and std::min respectively.   

{@link galois::GAccumulator} is used to accumulate a set of values. It provides {@link galois::GAccumulator::operator+=} and {@link galois::GAccumulator::operator-=} for adding or minusing new values.     
   
{@link galois::GSetAccumulator} is used to union sets 

{@link galois::GVectorAccumulator} is used to concatenate vectors.   

{@link galois::GListAccumulator} is used to splice std::list instances together.   
  
{@link galois::GVectorPerItemReduce} and {@link galois::GMapPerItemReduce} provides element-wise reduction over vector and map. {@link galois::GVectorPerItemReduce} reduces vectors where the reduction operator is element-wise binary operator of the type of the elements inside the vector. {@link galois::GMapPerItemReduce} reduces maps, each of which is a set of key-value pairs. The reduction operator reduces the values with the same keys.   

<hr>

@subsection collections Collections

<H4> @link galois::InsertBag </H4>

{@link galois::InsertBag} is an unordered collection allowing parallel insertions. One of its important characteristics is that it uses customized memory allocations to achieve scalable parallel insertions. 

  - Defining an {@link galois::InsertBag}

    {@link galois::InsertBag} has two template parameters:
     
     - T: Type of elements that the insertBag contains. 

     - BlockSize: The maximum memory that the elements to be inserted can take per thread. In other words, BlockSize is equal to the product of the size of one element and the maximum number of elements one thread will insert into the bag. This parameter is optional and the default value is 0, which stands for one memory page.   

    An example of defining a InsertBag can be seen in the {@link apps/delaunaytriangulation/DelaunayTriangulationDet.cpp}:

    @snippet apps/delaunaytriangulation/DelaunayTriangulationDet.cpp Define InsertBag 

  - Inserting elements
    
    {@link galois::InsertBag::emplace} is used to insert copies of elements. Its paramters will be passed to the constuctor of the type of the elements to create the element in the bag. {@link galois::InsertBag::push} is a shortcut of {@link galois::InsertBag::emplace} to call the copy or move constuctor of the type of the elements. 

    An example of inserting elements into a InsertBag can be seen in the {@link apps/delaunaytriangulation/DelaunayTriangulationDet.cpp}:

    @snippet apps/delaunaytriangulation/DelaunayTriangulationDet.cpp Insert elements into InsertBag 

  - Accessing elements 
    
    Accessing the elements is only allowed in serial. 

    {@link galois::InsertBag::begin} and {@link galois::InsertBag::end} are used to access the elements.  

    {@link galois::InsertBag::empty} is used to check if the bag is empty. 
  
    An example of accessing elements inside a InsertBag can be seen in the {@link apps/delaunaytriangulation/DelaunayTriangulationDet.cpp}:

    @snippet apps/delaunaytriangulation/DelaunayTriangulationDet.cpp Access elements of InsertBag 
 

<hr>

@subsection perthreadstorage PerThreadStorage

{@link galois::runtime::PerThreadStorage} provides the ability to store data per thread. 

 - Defining {@link galois::runtime::PerThreadStorage}

   The template parameter of {@link galois::runtime::PerThreadStorage} is the type of data to be stored. The parameters of the constructor of {@link galois::runtime::PerThreadStorage} are the parameters to construct the data to be stored. 

   An example to define a {@link galois::runtime::PerThreadStorage} can be seen as follows:

   In {@link apps/boruvka/BoruvkaMerge.cpp}
 
   @snippet apps/boruvka/BoruvkaMerge.cpp define per thread storage

 - Accessing {@link galois::runtime::PerThreadStorage}

   {@link galois::runtime::PerThreadStorage::getLocal} is used to access the data stored in the calling thread. There are two versions of this function. One is without any parameter and the other you can pass in a thread id. The latter one is a slightly optimized version when the user knows the thread id. {@link galois::runtime::PerThreadStorage::getRemote} is used to access the data stored in other threads by passing the thread id.

   An example to access a {@link galois::runtime::PerThreadStorage} can be seen as follows:

   In {@link apps/boruvka/BoruvkaMerge.cpp}
 
   @snippet apps/boruvka/BoruvkaMerge.cpp access perThreadStorage


<hr>

@subsection buildyourown Building Your Own Data Structures


*/
