Multithreading and Multiprocessing

Pre-requisites: Processes and Threads and multi_cpu_and_multi_core_systems

Multithreading

A program runs in a single process which can spawn multiple threads.

OS can schedule these threads for concurrent execution using scheduling.

It might be possible for each thread to be executed simultaneously on a separate cores of a multi-core CPU, though it depends on the implementation of multithreading library / programming language.
e.g. Python does not allow parallel execution of threads belonging to single process. For more info check here. C++, Java allows it.

Since threads share virtual memory address space with the parent process, multi-threading is only possible in shared memory architecture, like a multi-core CPU.
Additionally, we need to make use locking mechanisms to prevent multiple threads from writing to the same memory location simultaneously.

Multithreading libraries in c++: openmp, std::thread

Multiprocessing

Same program runs across more than one processes.
Such processes do not share virtual memory address space, so they can be run either on a distributed memory architecture, like a multi-cpu system.

On a shared memory architecture, the OS takes care of separating the virtual address space. The OS can also schedule each process to be executed on different cores of a multi-core CPU.

Code and data memory is different for each process, so no locking mechanism required.

In order share data from one process to another, we need to use In-process communication like MPI.

Multiprocessing libraray in c++: openmpi

Multithreading + Multiprocessing

In this case, same program run in multiple processes, each process spawning multiple threads.

In C++, possible to implement using openmpi and openmp/std::threads together.