← Back to Writeups
CUSTOMMediumReversing

Новелла (Novella)

XESXOR9/2/20263 min read
#reversing#custom#medium#disassembly_analysis#key_extraction#xor_decryption#linux

Новелла (Novella)

Platform: duckerz | Category: Reversing | Type: Challenge | Difficulty: Medium | OS: Linux | Author: XESXOR | Date: 2026-01-15 | Status: Solved Techniques: disassembly_analysis, key_extraction, xor_decryption

Summary

Task: ELF64 visual novel with encrypted.bin file containing the flag. Solution: Extract XOR key (0xa5 repeating) from .rodata section via disassembly, decrypt the file accounting for 8-byte size header.

Новелла (Novella) — DUCKERZ CTF

Description

"Давайте почитаем!" (Let's read!)

The task provides an archive novella.zip containing:

  • novella — ELF64 executable (a visual novel in Russian)
  • encrypted.bin — encrypted file with the flag

Analysis

Initial Examination

bash
$ file novella
novella: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked...

$ file encrypted.bin
encrypted.bin: data

$ ls -la encrypted.bin
-rw-r--r-- 1 user user 56 Jan 15 12:00 encrypted.bin

‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

The binary is an interactive visual novel in Russian. Running it shows a text interface with multiple choice options.

Finding the Decryption Function

Using strings to search for interesting strings:

bash
$ strings novella | grep -i flag
$ strings novella | grep -i dec
dec_flag

Found the dec_flag function — obviously responsible for decrypting the flag.

Disassembly

bash
$ objdump -d novella | grep -A 50 "<dec_flag>"

Analysis of the dec_flag function (address 0x1339) shows:

  1. File opening: the function opens encrypted.bin
  2. Header reading: first 8 bytes contain the data size (little-endian)
  3. XOR decryption: data is XORed with a key from the .rodata section ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

Extracting the XOR Key

bash
$ objdump -s -j .rodata novella | grep -A 2 4af0
 4af0 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5  ................

XOR key: 16 bytes of 0xa5 (repeating pattern).

Structure of encrypted.bin

Offset  Size  Description
------  ----  -----------
0x00    8     Header: data size (0x22 = 34 bytes, little-endian)
0x08    48    Encrypted data (aligned to 16 bytes)

Verification:

bash
$ xxd encrypted.bin | head -4
00000000: 2200 0000 0000 0000 xxxx xxxx xxxx xxxx  "...............

‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

First 8 bytes: 0x22 = 34 — size of decrypted data.

Solution

Decryption Script

python
#!/usr/bin/env python3
"""
Novella - DUCKERZ CTF
XOR decryption with extracted key
"""

def decrypt_novella(filepath='encrypted.bin'):
    with open(filepath, 'rb') as f:
        data = f.read()
    
    # Parse header (8 bytes, little-endian size)
    size = int.from_bytes(data[:8], 'little')
    print(f"[*] Data size from header: {size} bytes")
    
    # Skip 8-byte header, get encrypted data
    encrypted = data[8:]
    print(f"[*] Encrypted data length: {len(encrypted)} bytes")
    
    # XOR key: 0xa5 repeated
    key = 0xa5
    
    # Decrypt
    decrypted = bytes([b ^ key for b in encrypted])
    
    # Extract actual data (trim padding)
    result = decrypted[:size]
    
    return result
‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

if __name__ == '__main__':
    flag = decrypt_novella()
    print(f"\n[+] Flag: {flag.decode('utf-8')}")

Execution

bash
$ python3 solve.py
[*] Data size from header: 34 bytes
[*] Encrypted data length: 48 bytes

[+] Flag: DUCKERZ{REDACTED}

Alternative Approaches

  1. Dynamic analysis: run the binary under a debugger (gdb/radare2) and set a breakpoint on dec_flag, then extract decrypted data from memory

  2. Patching: modify the binary to call dec_flag and output the result

  3. Emulation: use Unicorn Engine to emulate the dec_flag function ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍

Lessons

  • Visual novels and games often contain hidden functions for handling secrets
  • XOR with a fixed key is a common pattern in CTF reverse challenges
  • File headers often contain metadata (size, magic bytes) that help understand the structure ‍​‌‌​​‌​​​‌‌​​‌​​​​‌‌​‌​​​​‌‌‌​​‌​​‌‌​​​‌​​‌‌‌​​​​​‌‌​‌‌​​​‌‌​​​​‍