/**
\page parallel_loop Parallel Loops
\tableofcontents
TODO: needs update

\section parallel_loop Parallel Loops

\subsection for_each For-Each Loop

 {@link galois::for_each} is the most commonly used loop construct to express
 parallelism. It provides speculative unordered execution of a set of active
 elements (or work items). A for_each can be invoked with the following parameters:

 - A pair of iterators [begin,end) that mark the set of initial active elements
 - The *operator*, which is a C++ functor containing the loop body code. 
 - The scheduling policy for work items, used by the internal worklist. It is
   specified using the {@link galois::wl} construct. 
 - Optional argument for specifying the loop name, using the {@link
   galois::loopname} construct


An example to use for_each in {@link apps/tutorial/SSSPsimple.cpp} can be seen as follows:
@snippet apps/tutorial/SSSPsimple.cpp for_each in SSSPsimple

 - {@link galois::for_each_ordered}
 
This is for ordered Galois Iterators. 


\subsection operator Operator

The loop body of a {@link galois::for_each} is specified as a C++ functor. We call
this the *operator* of the loop. The functor takes two arguments:

- The active element (or work item) to process in current iteration. This is picked
  by the runtime using the scheduling policy
- A reference to the runtime context of the loop {@link galois::UserContext}. This
  reference provides certain services needed by the loop body, e.g. the ability to add 
  new items to the internal worklist (maintained by the runtime), allocate
  iteration local objects, and suspend the parallel loop temporarily, etc.

An example to define an operator in {@link apps/tutorial/SSSPsimple.cpp} can be seen as follows:
@snippet apps/tutorial/SSSPsimple.cpp Operator in SSSPsimple

The operator must be *cautious*. \b TODO: add ref to glossary here

\subsubsection type_traits Type Traits

We can specify several flags as typedefs in the operator to turn on and off certain
features provided by the Galois runtime. Often turning off a feature saves the
runtime cost of providing that feature and results in
optimized execution:

 - \b tt_does_not_need_push

    Indicates the operator does not generate new work and push it on the worklist

 - \b tt_does_not_need_aborts

    Indicates the operator doesn't need support for speculation. All iterations
    will commit successfully

 - \b tt_needs_per_iter_alloc

    Indicates the operator may request the access to a per-iteration allocator
    \b TODO: add ref here

 - \b tt_does_not_need_stats

    Indicates the operator doesn't need to record its execution statistics.

 - \b tt_needs_parallel_break

    Indicates the operator may request the parallel loop to be suspended
    temporarily, at which point another function can be executed serially


\subsection scheduling Scheduling Policies

The scheduling policy for {@link galois::for_each} can be specified by choosing
one of the types in {@link galois::worklists} namespace and providing the type-name
as an argument to {@link galois::wl} parameter of the for_each. 
The scheduling policies are listed as follows:

 - \b Chunked Worklists

    Chunked worklists assign work items to threads one chunk at a time. Similarly,
    each thread accumulates new work in a chunk before putting it on the worklist. 
    Chunking offers better scalability because threads can amortize the cost of their access to
    the shared worklist over the entire chunk. The user chooses the size of the
    chunk: A large chunk size means less contention on the shared worklist, but may
    lead to load-imbalance, while a small chunk size may increase the contention on
    the shared worklist. The worklist of chunks itself can be organized in
    different ways, and we have observed that mapping the communication patterns of
    threads to the hardware-topology leads to more scalable implementations: 

      - \b ChunkFIFO (or ChunkLIFO) maintains a single global queue (or stack) for chunks of work items

      - \b PerSocketChunkFIFO (or PerSocketChunkLIFO) maintains a queue (or stack)
        of chunks per each socket (multi-core processor) in the system. A thread tries to find a chunk in its local
        socket before stealing from other sockets. 

      - \b PerThreadChunkFIFO (or PerThreadChunkLIFO) maintains a queue (or stack) of chunks per each thread. Normally
        threads steal work within their socket, but not every thread can steal
        from outside its socket. Each socket
        has a leader, which steals from other sockets when its own socket is out
        of work. 
        We show an example of using chunked worklists from {@link apps/delaunayrefinement/DelaunayRefinement.cpp}


 - \b OrderedByIntegerMetric
   
    Implements a priority queue based on a supplied function which maps a work
    item to an integer priority. Lower values are a higher priority. An inner
    queue may be passed to control how items within the same priority bin are stored. 
    An example to define an OrderedByIntegerMetric scheduling in {@link apps/tutorial/SSSPsimple.cpp} can be seen as follows:
    The OBIM in the following example shows how to define a priority scheduling. 
    The UpdateRequestIndexer defines the priority binning function. Internally It uses a PerSocketChunkLIFO to store the items with the same priority.

    @snippet apps/tutorial/SSSPsimple.cpp OrderedByIntegerMetic in SSSPsimple

    OBIM works well when the algorithms performance is sensitive to scheduling, and
    the work-items can be grouped into a small number of bins, ordered by integer
    priority (typically ~1000 bins). 


 - \b LocalQueues

    Creates local non-shared worklists which are used for all work generated during concurrent operation and use a global worklist for all initial work. 

 - \b BulkSynchronous

    Executes the work items in parallel rounds, where all existing items in the
    worklist are processed in current round, while any new items generated as a
    result of this processing are postponed till next round. The rounds are
    separated by barriers. 


 - \b OwnerComputes

    Is similar to LocalQueues, however, the user can provide a functor to map
    work items to threads. So a thread may generate a work item and specify another
    thread to process it. This functor is one of the template parameters, which
    maps each work item to the range [0:numThreads). Another
    difference from LocalQueues is that worklists are maintained per socket. 
    \b TODO: verify this


 - \b StableIterator

    Similar to {@link galois::do_all}, where the loop iterates over a fixed range
    of items, and the operator does not generate new work. The key difference from
    {@link galois::do_all} is that this scheduling policy supports speculation and
    allows for iterations to abort. 

 - \b OrderedList

    implements a std::pri_queue compatible priority queue. Elements of same
    priority are processed in arbitrary order. 

 - \b FIFO and \b LIFO
    
    Implements std::queue and std::stack like behaviors respectively. These
    implementations do not scale with the number of threads, and are often not
    suitable for parallel implementations. 


\subsection Deterministic for_each

{@link galois::for_each_det} is a deterministic parallel implementation of the
unordered for_each, described above. This implementation forces a deterministic
parallel schedule on the work-items. 

The operator must be *Cautious* (\b TODO link to glossary), just like the normal
for_each. The implementation executes the operator twice: in the first execution,
the operator is executed up till the *Fail-safe point*, which is first global write
in the operator. We call this the *Prefix* of the operator, and executing this
Prefix allows the runtime to compute the shared-data dependences. 
In the second execution, the operator is executed completely. The
user may specify the operator as two functors, where first contains the code for
the Prefix, while the second contains the remaining code for the operator. 

\b TODO: example of for_each_det



\subsection do_all Do-All Loop

{@link galois::do_all} is a loop construct for trivially parallel loops that do not
need any speculation, fancy scheduling policies, and do not generate new work items. The
threads iterate in parallel over a range of iterators specified in the invocation
of the do_all. The arguments are as follows:

- A pair of iterators specifying the set of work-items.

- A functor specifying the loop body of the do_all. It takes only one argument,
  which is the current work item. 

- {@link galois::steal} can be supplied with argument \b true or \b false to
  turn on work-stealing. This can be useful for loops where iterations have varying
  execution time. 

- Optionally {@link galois::loopname} can be used to specify a name of loop, which
  is useful for loop statistics

The file {@link apps/tutorial/HelloWorld.cpp} contains some examples of do_all
loops:
@snippet apps/tutorial/HelloWorld.cpp do_all example


\subsection on_each on_each loop

{@link galois::on_each} loop is for performing simple tasks on each thread in the
execution. The task is encoded in a functor passed to on_each loop. The functor
takes two arguments:

- \b tid: the thread-id assigned by runtime, which lies in the range
  [0:num-threads). 
- \b num-threads: the total number of threads used for execution

\b TODO: example of on_each
*/
