Equation solvers come in two flavors - direct and iterative. A direct solver is assured of finding the exact solution (modulo roundoff error) in a known (and finite) amount of time. An iterative solver has the property of continuously improving an estimate to the solution until a desired amount of accuracy is achieved. The PCG algorithm actually falls in both of these categories, but is used exclusively as an iterative solver.
For this project a PCG solver was added to FEAP. The vast majority
of the code was left in serial form, with only the actual PCG
algorithm, along with the requisite data distribution and initialization,
being implemented in parallel. The matrix 
and
the residual 
were formed by the elements and assembled by FEAP
serially, on the master node, requiring that the matrix be
distributed to the slave nodes.
The computational demands of the solver are primarily vector dot products and matrix-vector multiplies. The dot product requires the calculation of the local dot product (a dot product on the portion of the vector owned by the processor), and an all to all reduction (add) operation to accumulate the local results from all of the processors and send the final result back to all of the processors. The matrix-vector multiply requires the communication of a portion of the vector owned by a processor which is used by matrix data owned by other processors. A local matrix-vector multiply can then be carried out, and the portion of the result owned by other processors sent to those processors to be added to their result.