Live briefing
Everything about Data parallelism
Parallelization across multiple processors in parallel computing environments
Data parallelism is parallelization across multiple processors in parallel computing environments. It focuses on distributing the data across different nodes, which operate on the data in parallel. It can be applied on regular data structures like arrays and matrices by working on each element in parallel. It contrasts to task parallelism as another form of parallelism. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Data parallelismA 5-minute orientation
- Skim a recent paper: Accelerating Federated Learning With Data and Model Parallelism in Edge ComputingInfluential work since 2022
- Pick up a classic book: Parallel Processing and Applied MathematicsThe most-reprinted book on the subject
- Read the landmark paper: Partitioning functions for stateful data parallelism in stream processingCited 103 times
- Go sideways: Ateji PXA closely related topic
- Go sideways: Multidimensional DSP with GPU accelerationA closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🏆 Landmark papers (most cited)
Partitioning functions for stateful data parallelism in stream processing
Continuous retrieval of multimedia data using parallelism
Approaches for integrating task and data parallelism
Data and task parallelism in ILP using MapReduce
Safe Data Parallelism for General Streaming
Exploiting instruction- and data-level parallelism
Function level parallelism driven by data dependencies
Exploiting task and data parallelism on a multicomputer
🆕 Influential since 2022
Accelerating Federated Learning With Data and Model Parallelism in Edge Computing
Quantum data parallelism in quantum neural networks
HiHGNN: Accelerating HGNNs Through Parallelism and Data Reusability Exploitation
An efficient algorithm for data parallelism based on stochastic optimization
Optimistic Data Parallelism for FPGA-Accelerated Sketching
📚 Books worth reading
Parallel Processing and Applied MathematicsRoman Wyrzykowski · 2003
Parallel Computing TechnologiesVictor Malyshkin · 2003
HadoopTom White, Tom White · 2009
PerceptronsMarvin Minsky, Seymour Papert · 1969
Using MPIWilliam Gropp, Ewing Lusk · 1994
Distributed SystemsAndrew S. Tanenbaum, Maarten Van Steen · 2001
Real-time embedded multithreadingEdward L. Lamie · 2005
Parallel system interconnections and communicationsMiltos D. Grammatikakis, D. Frank Hsu · 2000
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Ateji PX Ateji PX is an object-oriented programming language extension for Java. It is intended to facilliate parallel computing on multi-core processors, GPU,…
- Multidimensional DSP with GPU acceleration Multidimensional digital signal processing (MDSP) refers to the extension of digital signal processing (DSP) techniques to signals that vary in more than…
- Array programming In computer science, array programming refers to solutions that allow the application of operations to an entire set of values at once. Such solutions…
- Futhark (programming language) Futhark is a multi-paradigm, high-level, functional, data parallel, array programming language. It is a dialect of the language ML, originally developed…
- General-purpose computing on graphics processing units General-purpose computing on graphics processing units (GPGPU, or less often GPGP) is the use of a graphics processing unit (GPU), which typically handles…
- Systolic array In parallel computer architectures, a systolic array is a homogeneous network of tightly coupled data processing units (DPUs) called cells or nodes. Each…
- OpenMP OpenMP is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on…
- Stream processing In computer science, stream processing (also known as event stream processing, data stream processing, or distributed stream processing) is a programming…