HPC!
CCP-TEPP SummerSchool 2026
UKRI - STFC
Preamble
Disclaimers
This is a scattershot of content from the following (in development courses) with some additions…
Goals!
- Understand how language choice and platform affect decisions in implementation
- Run an mpi code to see Amdahls law in action
- Know what strong and weak scaling mean
Cache
Computer Model (concept)
Dot product in a computer
Computer model (actual)
Memory layout
Matrix multiplication
Exercise 1
- Download and unpack https://carpentries-incubator.github.io/HPC_Advanced_User-Teaching-performance-programming-to-scientists/data/code.zip
- Check the effect of cache (in a language of your choice)
Challenge 1 *(c or fortran example)
How does optimisation affect matmult, matmult_swapped, and matmult_blas?
Flops
Scaling
Parallel computing
- If two results don’t depend on each other, the can be done at the same time!
\[ D_\text{prod} = \left(\sum_{i=0}^{N/2 - 1} X_i * Y_i\right) + \left(\sum_{i=N/2}^{N - 1} X_i * Y_i\right) \]
Speedup
The standard way of expressing the improvement is
\[ \text{speedup} = \frac{\text{time without optimisation}}{\text{time with optimisation}} \]
Strong scaling
How does parallelism affect the time to complete a fixed input?
Why don’t we always get perfect scaling?
Exercise 2
Let’s see this in action with an emulation.
pixi initpixi add python mpi4pypixi add --pypi amdahlpixi run mpiexec -n 2 amdahl -w 10 -p 0.5
Challenge 2
- Run for a range of number of processors for
-p 0.9. - Note the speedup as you increase the number of processors.
- What trend are you seeing?
- What about for
-p 0.25?
Amdahls law
\[ \text{s} = \frac{1}{(1-p) + \frac{p}{N}} \] where \(s\) is the overall speedup, \(p\) is the fraction of the program that can be paralellised, and \(N\) is the number of parallel processes.
- \(p=0.9\), \(N=4\) -> \(s=3.08\)
- \(p=0.25\), \(N=2\) -> \(s=1.14\)
Weak scaling
How does parallelism affect the time if the problem size increases proportionally?
\[ D_\text{prod} = \sum_{i=0}^{N - 1} X_i * Y_i \]
Challenge 3
- Perform a weak scaling test by increasing the work proportionally to the number of processors for
p = 0.9andp = 0.25 - Scale the runtime by the size of the problem before calculating your speedup
Gustafsons law
\[ \text{s} = (1-p) + p*N \] where \(s\) is the overall speedup, \(p\) is the fraction of the program that can be paralellised, and \(N\) is the number of parallel processes (and the scale of the problem size).
Choosing Precision
Float representation
The highs and lows
Low precision
❌ Large error
✔️ High throughput (flops/second)
✔️ Low memory requirements
✔️ Low energy requirements
High precision
✔️ Small error
❌ Low throughput
❌ High memory requirements
❌ High energy requirements
Exercise 3
Go back to the matmult_blas code. Compare what happend when you use sgemm and float instead of dgemm and double.
Final comments
- Check that arrays are accessed contiguously
- Parallelism can only help with the parallel bits!
- If you don’t need the accuracy, don’t compute it
