Writing Your First Buffer Overflow Exploit: A Beginner's Guide to Stack Smashing
A crash is only a curiosity until you can steer it. The moment a debugger shows four bytes of your own input sitting in EIP, a segmentation fault stops being noise and becomes something you can aim. What follows takes you from a deliberately broken 32-bit program to a live reverse shell—inside an isolated lab, on a machine you own, with every step explained. The point is not to break the internet. The point is to understand why memory safety bugs have shaped decades of exploitation and defense.
Modern 64-bit systems are the real world, but 32-bit stack smashing remains the clearest classroom. The calling convention is simpler, the register set is smaller, and the stack frame is easier to visualize. In a classic 32-bit x86 program, a function call pushes a return address onto the stack. The called function then saves the previous frame pointer, allocates space for local variables, and runs its code. If one of those local variables is a fixed-size character buffer, and the program copies more data into it than it can hold, the extra bytes spill upward into saved frame data and eventually into the saved return address.
That saved return address is the target. When the function finishes, the CPU executes a ret instruction, which pops that saved address back into EIP—the instruction pointer. If you have overwritten it, the CPU will try to execute whatever address you supplied. That is the entire trick: turn a dumb overflow into control of the next instruction. Four bytes of input become a steering wheel.
Picture a vulnerable function such as void vuln(char *input) { char buffer[64]; strcpy(buffer, input); }. The strcpy call does not check length. If you pass 100 bytes, the first 64 fill the buffer, the next 4 overwrite the saved frame pointer, and the next 4 overwrite the saved return address. On a 32-bit little-endian system, an address like 0xffffd610 is written into memory as the bytes 10 d6 ff ff. This byte order matters when you craft your payload.
Before you write a single byte, confirm the layout in a debugger. Set a breakpoint just before the function returns, then inspect the stack with info registers, x/20x $esp, and bt. You are looking for the distance between the start of your buffer and the saved return address. That distance is your offset.
Do this only in a virtual machine or an authorized training environment. Use a host-only network, take a snapshot before you begin, and never point exploit code at a system you do not own or have explicit permission to test. Disable ASLR temporarily for reproducibility with a command such as echo 0 | sudo tee /proc/sys/kernel/randomize_va_space, then re-enable it afterward. You will also want gcc, gdb, a GDB extension like pwndbg or gef, Python for payload generation, and netcat for a listener.
Compile the target with protections turned off so the lesson is visible. A typical command looks like gcc -m32 -g -fno-stack-protector -z execstack -no-pie vulnerable.c -o vulnerable. The -m32 flag builds a 32-bit binary. -fno-stack-protector removes the canary that would otherwise detect the overflow. -z execstack makes the stack executable, which allows shellcode placed there to run. -no-pie keeps addresses fixed, so you can predict where the stack lives. In a real hardened system, none of these conditions would be true by default—and that is precisely why this exercise is a lesson, not a recipe.
Run the program with a long string of characters, for example a hundred A bytes. It should segfault. Open it in GDB, run it again with the same input, and let it crash. Now type info registers. If EIP contains 0x41414141, you have proven the overflow reaches the return address. The letter A is 0x41 in hexadecimal, so four A's in EIP mean four bytes of your input are now the instruction pointer. This is the “aha” moment. From here, the exploit is an engineering problem: find the exact offset, choose a destination address, and place a payload there.
Do not guess the offset by adding A's until the crash changes. Use a non-repeating cyclic pattern instead. A de Bruijn sequence gives every four-byte sequence a unique value, so when the program crashes, the value in EIP tells you exactly where in the pattern the return address was overwritten. Tools such as pattern_create and pattern_offset, or the cyclic and cyclic_find functions in pwntools, make this quick. In a simple 32-bit stack frame, the offset to EIP is often the buffer size plus four bytes for the saved frame pointer, but you must verify it in your own binary. The debugger is the source of truth.
Once you know the offset, your payload starts to take shape: padding of exactly that length, followed by the address you want EIP to hold, followed by a NOP sled and your shellcode. The NOP sled is a run of 0x90 bytes that gives the CPU a safe landing zone. If EIP points anywhere in that sled, execution slides forward into the shellcode. To find a useful address, break just before the vulnerable function returns and examine $esp. After the ret, ESP often points directly at the bytes following the return address—your NOP sled. That stack address is a common first target, though stack layout can shift slightly depending on environment variables and arguments.
A reverse shell is simply a payload that connects back to a listener you control. Instead of binding a shell to a port on the target, it reaches out to your machine, which is often more useful in lab networks and firewalled environments. Generate the payload with a framework or hand-written assembly, but keep it small and free of bad characters such as null bytes. If the buffer is only a few hundred bytes, size matters. Test first with a payload that spawns a local shell, then switch to the reverse shell once the control flow is reliable. Start a listener with nc -lvnp 4444 on your lab host, run the exploit, and watch for the connection. If it fails, check the offset, the address endianness, the bad characters, and the stack alignment. Exploit development is mostly debugging.
Modern systems stack several mitigations against exactly this technique. Stack canaries place a secret value between the buffer and the return address; if the canary changes, the program aborts before returning. ASLR randomizes stack, heap, and library addresses, so your hard-coded jump target is unlikely to be correct. NX and DEP mark the stack non-executable, so shellcode placed there will not run. PIE randomizes the main executable, and 64-bit calling conventions change the register and stack layout. Newer CPUs add control-flow integrity features. A real-world exploit against a modern target usually needs an information leak, a return-oriented programming chain, or a bypass for each mitigation. That complexity is not a reason to skip the basics—it is the reason the basics matter.
Use CTFs, exploit education platforms, and intentionally vulnerable virtual machines. Read classic papers such as “Smashing the Stack for Fun and Profit,” work through books like Hacking: The Art of Exploitation, and rebuild the same exercise on 64-bit Linux to see how much changes. Always stay inside systems you own or have written authorization to test. The crash is not the goal. The controlled instruction pointer is.
Understanding the overflow teaches you why the defense exists—and why memory safety still matters.
Photo: Sergei Starostin / Pexels
Previous Post
Common Mistakes in Exploit Development: Off-by-One, Bad Shellcode, and Unstable Payloads
Next Post
How to Set Up a Safe Malware Analysis Lab with Virtual Machines and Snapshots
