Concurrency and Parallel Computing Explained

Concurrency and Parallel Computing

Concurrency is about structuring a program as several tasks whose lifetimes overlap — handling many requests, waiting on many sockets, reacting to events. Parallelism is about running computations at literally the same time on several cores to finish sooner. The two often appear together, but they solve different problems: concurrency is mostly a correctness and design challenge, parallelism mostly a performance one.

Both are hard for the same reason: as soon as two flows of execution share mutable state, the order in which their operations interleave becomes part of the program's behavior. Bugs that depend on timing are rare in testing and common in production, where load, core counts, and hardware memory models differ. This guide covers the mechanisms and the disciplines that keep concurrent code correct and parallel code fast.

TL;DR

Concurrency structures overlapping tasks; parallelism executes work simultaneously. An event loop is concurrent on one core; a parallel matrix multiply needs many.
A data race — two threads accessing the same memory, at least one writing, without synchronization — is a bug in every language, and undefined behavior in C, C++, and Go.
Protect shared state with one discipline: locks, atomics, or ownership transfer through channels. Mixing them ad hoc is where bugs live.
Deadlock needs a cycle of threads waiting on each other's locks; a global lock order prevents it.
Amdahl's law caps speedup by the serial fraction: a program that is 10% serial never runs more than 10x faster, however many cores you add.
Use the tools: race detectors, thread sanitizers, and stress tests find bugs that code review misses.

Quick Example

A thousand goroutines increment a shared counter. The first version has a data race; the second is correct.

Running go run -race main.go reports WARNING: DATA RACE for the first loop — the race detector finds the bug even on runs where the total happens to come out right.

Core Concepts

Concurrency Versus Parallelism

Race Conditions and Data Races

A race condition is any bug where the result depends on timing — for example, two requests both reading a balance, both deciding there is enough money, and both withdrawing. A data race is the low-level version: unsynchronized concurrent access to the same memory with at least one write. Data races are always bugs; race conditions can also happen with perfectly synchronized code (and across processes and databases) when the logic assumes an ordering that is not guaranteed.

The classic shape is check-then-act: read some state, decide, then act on the decision while another thread changes the state in between.

Synchronization Primitives

Memory Models and Ordering

Compilers and CPUs reorder memory operations for speed. Within one thread the result is indistinguishable from program order, but other threads can observe surprising orders — especially on ARM and other weakly ordered architectures. A language's memory model defines which orders are allowed and what synchronization establishes a happens-before relationship.

The practical guarantee most languages give is "sequential consistency for data-race-free programs": if all shared accesses are properly synchronized, the program behaves as if operations interleaved in some single order. Atomics with weaker orderings (relaxed, acquire/release in C++ and Rust) are for experts building lock-free structures; default to sequentially consistent atomics or locks.

Deadlock, Livelock, and Starvation

Deadlock requires four conditions at once — mutual exclusion, hold-and-wait, no preemption, and a circular wait. Break any one and deadlock is impossible; the usual choice is a global lock order, so every thread acquires locks in the same sequence. Livelock is threads repeatedly reacting to each other without progress; starvation is a thread that never gets the resource because others keep winning.

Concurrency Models

Threads and locks — shared memory, explicit synchronization. Flexible, error-prone.
Message passing — communicating sequential processes (Go channels) or actors (Erlang, Akka): share by communicating instead of communicating by sharing.
Async/await and event loops — many tasks on few threads, cooperatively yielding at await points. Excellent for I/O; one blocking call stalls everything. See Asynchronous JavaScript and Rust Async.
Lightweight threads — runtime-scheduled tasks with a blocking programming model: goroutines and Java virtual threads.
Data parallelism — the same operation over many elements: parallel loops, SIMD, and GPUs.

Parallel Performance

Amdahl's Law

If a fraction p of the work parallelizes perfectly over N cores, the best possible speedup is:

The serial fraction dominates quickly. Before adding cores, find and shrink the serial parts: a single lock, a sequential merge step, a shared queue.

Gustafson's Law

Amdahl assumes a fixed problem size. In practice, more cores are often used to solve bigger problems in the same time, where the parallel part grows with the data and scaling looks far better. Both views are useful: Amdahl for latency targets, Gustafson for throughput and data size.

What Erodes Parallel Speedup

Contention — threads queueing for the same lock or atomic.
False sharing — independent variables sharing a cache line, see Computer Architecture.
Load imbalance — one slow partition holds up the whole job; use work stealing or smaller chunks.
Oversubscription — more runnable threads than cores, adding context switches without adding throughput.
Memory bandwidth — many cores streaming data can saturate memory before they saturate compute.

Best Practices

Prefer Not Sharing

The easiest shared state to get right is none. Give each task its own data and combine results at the end (map then reduce), pass ownership through channels, or use immutable data that can be read without locks.

Keep Critical Sections Small and Free of I/O

Hold a lock only long enough to update in-memory state. Never perform network calls, disk I/O, or callbacks into unknown code while holding it.

Size Pools to the Work

For CPU-bound work, roughly one thread per core. For blocking I/O, more threads or — better — async I/O or lightweight threads. Bound every pool and queue so overload produces back-pressure instead of memory exhaustion.

Use Higher-Level Constructs

Thread pools, concurrent collections, parallel streams, asyncio.gather, errgroup, and structured concurrency scopes encode correct patterns. Hand-rolled synchronization should be rare and reviewed carefully.

Run Race Detectors in CI

Go's -race, ThreadSanitizer for C/C++/Rust, and stress tests with randomized scheduling catch races before production does.

Common Mistakes

Check-Then-Act Without Atomicity

Inconsistent Lock Ordering

Blocking the Event Loop

Assuming More Threads Means More Speed

FAQ

What is the difference between concurrency and parallelism?

Concurrency means multiple tasks are in progress at overlapping times; parallelism means multiple computations execute at the same instant on different cores. A single-threaded event loop serving thousands of connections is concurrent but not parallel. A matrix multiply split across 16 cores is parallel.

What is a data race?

Two threads access the same memory location at the same time, at least one of them writes, and nothing orders the accesses. The result can be lost updates, torn values, or — in C, C++, and Go — undefined behavior. Race detectors find data races by tracking happens-before relationships at runtime.

How many threads should my program use?

For CPU-bound work, about one per core; more just adds switching. For I/O-bound work, the answer depends on how long threads block, which is why async I/O and lightweight threads (goroutines, virtual threads) are popular: they let you have thousands of concurrent tasks without thousands of OS threads.

Is async/await the same as multithreading?

No. Async/await lets one thread interleave many tasks by suspending each at await points while it waits for I/O. It provides concurrency, not parallelism, unless the runtime also spreads tasks across a thread pool. CPU-heavy work inside an async task still blocks every other task on that thread.

Why didn't adding cores make my program faster?

Usually a serial bottleneck (Amdahl's law), lock contention, false sharing, or a resource other than CPU — memory bandwidth, disk, network, or a downstream service. Profile first: a profiler and lock-contention metrics usually point straight at the limit.