# Definition of "embarrassingly parallel"

## embarrassingly parallel

### adjective

*comparative*more embarrassingly parallel,*superlative*most embarrassingly parallel(programming) In parallel computing, of a particular problem: very easy to separate into a number of parallel tasks.

### Quotations

We call such applications "

" to emphasize the fact that, while there is a high degree of parallelism and it is possible to make efficient use of many processors, the granularity is large enough that no cooperation between the processors is required within the matrix computations.**embarrassingly parallel**1986, Cleve Moler, “Matrix Computation on Distributed Memory Multiprocessors”, in Michael T[homas] Heath, editor, Hypercube Multiprocessors 1986, Philadelphia, Pa.: Society for Industrial and Applied Mathematics, part IV (Numerical Computations), page 182

In the product algorithm, in fact, only one arithmetic operation, a multiplication, is required, so with a sufficient number of processors it executes in constant time regardless of the degree of the result. […] This is the "

" aspect of the algorithms: doing this part in parallel is so trivial it's hardly worth mentioning.**embarrassingly parallel**1992, Dennis Weeks, “Embarrassingly Parallel Algorithms for Algebraic Number Arithmetic”, in Richard E. Zippel, editor, Computer Algebra and Parallelism: Second International Workshop, Ithaca, USA, May 9–11, 1990: Proceedings (Lecture Notes in Computer Science; 584), Berlin; Heidelberg, Baden-Württemberg: Springer-Verlag, page 63

Both of the example applications we've looked at here would be considered

. […] By contrast, most parallel sorting algorithms require a great deal of interaction. For instance, consider merge sort, a common method of sorting numbers. It breaks the vector to be sorted into two (or more) independent parts, say the left half and right half, which are then sorted in parallel by two processes. So far, this is**embarrassingly parallel**, at least after the vector is divided in half. But then the two sorted halves must be merged to produce the sorted version of the original vector, and that process is not**embarrassingly parallel**.**embarrassingly parallel**2011, Norman Matloff, “Parallel R”, in The Art of R Programming: A Tour of Statistical Software Design, San Francisco, Calif.: No Starch Press, page 347

We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines. […] [W]e make use of the recently proposed nonparametric VI to facilitate an

VI procedure that can be applied to a wider scope of models, including to nonconjugate models. We derive our**embarrassingly parallel**VI algorithm, analyze our method theoretically, and demonstrate our method empirically on a few nonconjugate models.**embarrassingly parallel**2015 October 14, Willie Neiswanger, Chong Wang, Eric Xing, “Embarrassingly Parallel Variational Inference in Nonconjugate Models”, in arXiv, Ithaca, N.Y.: Cornell University, abstract