Computing Fundamentals

Every abstraction you work with eventually lands on the same machinery: a processor fetching instructions, a memory hierarchy that is fast only when you are predictable, an operating system multiplexing many programs onto a few cores, and bytes that only mean something once both sides agree on an encoding. You can ship a lot of software without thinking about any of it — until a loop that "should" be fast is ten times slower than expected, an emoji corrupts a database column, or adding threads makes a service slower.

This topic collects the fundamentals that explain those surprises. It is written for developers rather than hardware engineers: each guide starts from a symptom you have probably seen, explains the mechanism underneath, and ends with what to do differently in code.

TL;DR

• Computer architecture decides real performance: a cache miss to main memory costs roughly a hundred times more than a hit, so data layout often matters more than instruction count.
• Operating systems give each program the illusion of its own machine through processes, threads, scheduling, and virtual memory — and every illusion has a cost you can measure.
• Binary data and character encoding explain integer overflow, 0.1 + 0.2 != 0.3, byte order, and why "string length" has at least three answers.
• Concurrency and parallelism are different problems: concurrency is about structure and correctness, parallelism is about speed, and Amdahl's law caps the speedup.
• GPU computing trades latency for throughput: thousands of simple threads win on data-parallel arithmetic and lose on branchy, sequential work.

From Silicon to Your Program

Each layer is a contract. Performance problems and data corruption bugs usually live where a program quietly relies on something the contract does not promise.

Featured Topics

How the Machine Runs Code

• Computer Architecture — Instruction execution, pipelines, caches, the memory hierarchy, branch prediction, and I/O
• Operating System Concepts — Processes, threads, scheduling, virtual memory, file systems, and system calls
• Computability and Complexity Classes — The limits of computation: halting, P vs NP, and hard problems in practice

How Data Is Represented

• Binary Data and Character Encoding — Integers, floating point, byte order, Unicode, UTF-8, and serialization formats
• Probability and Linear Algebra for Computing — The probability, statistics, vectors, and matrices engineers actually use
• Discrete Mathematics for Developers — Logic, sets, relations, combinatorics, graphs, and modular arithmetic in code

Doing Many Things at Once

• Concurrency and Parallel Computing — Races, locks, atomics, memory ordering, message passing, and parallel speedup
• GPU Computing and CUDA — GPU architecture, the CUDA programming model, memory coalescing, and how ML workloads use accelerators

Common Questions

Do application developers really need this?

You need enough to recognize the symptoms. Most production performance problems are not algorithmic — they are cache misses, lock contention, excessive system calls, or data being copied between processes and devices. Knowing the mechanism turns "it's slow" into a hypothesis you can measure with a profiler. You do not need to design a CPU; you do need to know why iterating an array beats chasing pointers.

Where should I start?

Start with Computer Architecture for the memory hierarchy, then Operating System Concepts for processes and virtual memory. Those two explain most performance behavior. Read Binary Data and Character Encoding before your first internationalized product, and Concurrency and Parallel Computing before your first multi-threaded service.

How does this relate to systems programming?

Systems programming applies these fundamentals in languages such as C and Rust that expose memory and threads directly. The fundamentals apply to every language, including garbage-collected and interpreted ones — a Python service still pays for cache misses, page faults, and context switches.

Is this the same as a computer science degree?

It overlaps with the systems half of one, minus the proofs and circuit design. The emphasis here is practical: what to measure, what to change, and which mental model explains the behavior you are seeing. For algorithmic foundations, see Algorithms & Data Structures.

Learning Path

Beginner

Learn the memory hierarchy, the difference between a process and a thread, and how UTF-8 encodes text.

Intermediate

Profile a hot loop and explain its cache behavior, read strace output for a slow program, and find a data race with a race detector (Concurrency).

Advanced

Reason about memory ordering and lock-free code, tune NUMA and thread placement, and write and profile a GPU kernel.

Related Topics

• Systems Programming — Applying these fundamentals in C, C++, and Rust
• Compilers & Language Implementation — How source code becomes the instructions described here
• Performance & Optimization — Measuring and fixing the behavior these guides explain
• Linux — The operating system most servers run
• Algorithms & Data Structures — The algorithmic side of performance