The Cursed Syscall

Reverse Engineering, Medium, 300 pts

FLAG{c4pt41n_fl1nt_s_curs3d_c0mp4ss_st33rs_t0_tru7h} (verified offline against the binary's filter)

The binary is a stripped 16 KB x86-64 PIE. main at 0x11d0 reads a line with fgets, strips the newline, and requires strlen(input) == 0x34 (52). The 52 bytes are copied to the stack and treated as thirteen little-endian 32-bit chunks.

It then calls prctl(PR_SET_NO_NEW_PRIVS, 1) and prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog). The sock_fprog at vaddr 0x4620 is {len=192, filter=0x4020}; the BPF program is 192 classic sock_filter instructions stored in .data.

After installing the filter the program loops. The counter starts at 1, is incremented before the syscall, and is compared with 13 after it, so exactly 12 iterations run. A compiler-generated divide-by-13 (magic 0x4ec4ec4f, shift 34) selects chunks with wraparound. Iteration s passes four consecutive chunks (v[s], v[s+1], v[s+2 mod 13], v[s+3 mod 13]) as the arguments to syscall(1718, ...).

Syscall 1718 (0x6b6) does not exist. The kernel runs the seccomp filter before rejecting the number, and that filter is the actual checker. On a match it returns SECCOMP_RET_ERRNO with data 0, so the syscall returns 0 and errno stays 0, which is what the program checks. On a mismatch it returns SECCOMP_RET_ERRNO with data 1, the program sees a nonzero errno and prints "Ye walk the plank". Normal syscalls are allowed.

Decoding the filter (work/bpf2.py): if nr != 0x6b6, allow. Otherwise require every byte of args[0] to be below 0x80, then compute from the four 32-bit arguments:

M0 = a0
M1 = a1 ^ M0
M2 = a2 ^ M1
M3 = a3 ^ M2
M4 = M0 + M1 + M2 + M3
M5 = M0 + M1 - M2 - M3
M6 = M0 - M1 + M2 - M3
M7 = M0 - M1 - M2 + M3
M8  = (M6 | M7) ^ (M4 & M5)
M9  = (M7 | M4) ^ (M5 & M6)
M10 = (M4 | M5) ^ (M6 & M7)
M11 = (M5 | M6) ^ (M7 & M4)

and compare (M8, M9, M10, M11) against twelve hard-coded 4-tuples. Only those twelve values get ERRNO(0); everything else gets ERRNO(1).

The program's twelve calls are sliding windows over the same thirteen chunks, so the correct 52-character phrase has to make every window match one of the twelve tuples. With the known flag format, the phrase was solved with z3 (work/z3chain.py): thirteen 32-bit unknowns restricted to printable ASCII, v[0] == "FLAG", byte 4 '{', byte 51 '}', and each of the twelve windows constrained to equal one of the filter tuples. The first model already produced the readable prefix FLAG{c4pt41n_fl1nt_s_curs3d_c0mp4ss_st33rs_t; fixing that prefix and the closing brace left exactly one tail, 0_tru7h}, from work/tail.py.

Verification (work/verify.py): reimplementing the program's argument schedule and the BPF filter in Python, the phrase passes all twelve calls with errno 0. Each call hits a different one of the twelve tuples, exactly once, so the windows are a bijection onto the filter's allow-list. That is strong evidence this is the intended phrase and not just another satisfiable input.

Flag: FLAG{c4pt41n_fl1nt_s_curs3d_c0mp4ss_st33rs_t0_tru7h}