ASSEMBLY / CHAPTER 13

Nested calls and System V stack discipline

Master

Preserve needed state across an arbitrary conforming callee.

By the end of this chapter

  • Align the stack before a call
  • Save callee-saved registers when used
  • Restore the stack before returning

The idea

This chapter uses the ordinary System V AMD64 convention for Linux integer-only examples. On entry after an ordinary call, RSP is eight bytes away from a sixteen-byte boundary because the return address has been pushed. Before making another ordinary call, arrange sixteen-byte alignment. Some wider vector or stack-argument cases impose additional requirements and are outside this example.

A helper may overwrite caller-saved registers, so values needed afterward must live in preserved storage. RBX is callee-saved: using it locally requires saving its incoming value and restoring that value before return. The example’s single push both preserves RBX and supplies the needed alignment. Match every stack adjustment, keep the return address intact, and do not treat a calling convention from another operating system as interchangeable.

Assembly example
; Conceptual System V AMD64 function; ordinary integer-only calls.
; Entry RSP mod 16 = 8. Helper returns 10 and preserves RBX.
; For RDI=7, return helper() + original RDI.
combined:
push rbx
mov rbx, rdi
call helper
add rax, rbx
pop rbx
ret
Result
RAX = 17; incoming RBX is restored; the stack is balanced.

PUSH changes alignment by eight bytes, RBX retains the needed argument across helper, and POP restores the caller’s RBX.

Try it yourself

Write a nested-call skeleton that saves two callee-saved registers and adds any further alignment adjustment needed before calling.

Use your own editor for this exercise. Runnable challenges on CodeQuest are available for JavaScript, Python, and SQL.

CHECK YOUR UNDERSTANDING

Chapter checkpoint

With entry RSP mod 16 = 8, what is RSP mod 16 immediately after push rbx?