Tag: Metadata Extraction

GotMeta

A Python tool for discovering, downloading, and extracting metadata from publicly available files on target domains. Useful for OSINT.

Features

• File Finder – Find publicly available files on target domains
• Metadata Extraction – Extract all metadata using exiftool
• Multi-threaded Downloads – Fast concurrent file downloads (default 5 threads)
• Proxy Support – Route through Burp Suite or other proxies to bypass anti-bot protection
• URL List Mode – Skip API search and download from manual file list.

What Metadata Can Reveal

• Usernames – Author, Creator, LastModifiedBy
• Email addresses – Company
• Software versions – Application, Producer
• Internal file paths – SourceFile paths may reveal folder structure
• Computer names – May appear in various fields
• Creation dates – Document timeline
• GPS coordinates – Photos/images (if targeting image files)

Google API Quota

• Free tier: 100 queries/day
• Each file type search uses ~1-10 queries depending on results

If you run out of quota, use --url-list mode with manually collected URLs

Installation

[pastacode lang=”markup” manual=”%23%20Clone%20the%20repository%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2FMrTurvey%2FGotMeta.git%0A%20%20%20cd%20GotMeta%0A%0A%23%20Install%20dependencies%0A%20%20%20pip%20install%20requests%0A%0A%23%20Download%20exiftool%20for%20metadata%20extraction%0A%20%20%20Windows%3A%20https%3A%2F%2Fexiftool.org%2F%0A%20%20%20Linux%3A%20sudo%20apt-get%20install%20exiftool%0A%20%20%20Mac%3A%20brew%20install%20exiftool” message=”” highlight=”” provider=”manual”/]

Quick Start

Step 1: Create Google Cloud Project

• Go to Google Cloud Console (https://cloud.google.com/)
• Login with a Google account
• Click “Select a project” → “New Project”
• Enter a project name (e.g., “GotMeta”)
• Click “Create”

Step 2: Enable Custom Search API

• In your project, go to “APIs & Services” → “Library”
• Search for “Custom Search API”
• Click on it and press “Enable”

Step 3: Create API Key

• Go to “APIs & Services” → “Credentials”
• Click “Create Credentials” → “API Key”
• Copy your API key

Step 4: Create Custom Search Engine

• Go to Google Programmable Search Engine (https://programmablesearchengine.google.com)
• Click “Add a search engine” -Enter any name (e.g., “GotMeta Search”) -For “Sites to search”, select “Search the entire web”
• Click “Create”
• Copy your Search Engine ID (looks like: 94e6b2cd62344413c)

Basic Usage

[pastacode lang=”markup” manual=”%23%20Search%20and%20download%20files%20from%20a%20domain%20and%20extract%20all%20metadata%0Apython%20-u%20GotMeta.py%20-d%20example.com%20-k%20YOUR_API_KEY%20-s%20YOUR_SEARCH_ENGINE_ID%20–download%20–extract-all%20metadata.csv%0A%0A%23%20Bypass%20Anti-Bot%20Protection%20via%20Proxy%20(see%20README%20section%20below)%0Apython%20-u%20GotMeta.py%20-d%20example.com%20-k%20YOUR_API_KEY%20-s%20YOUR_SEARCH_ENGINE_ID%20–download%20–extract-all%20metadata.csv%20–proxy%20http%3A%2F%2F127.0.0.1%3A8080%0A%0A%23%20Use%20a%20pre-compiled%20URL%20list%20(no%20API%20required)%20to%20download%20and%20extract%20all%20metadata%0Apython%20-u%20GotMeta.py%20–url-list%20urls.txt%20–download%20–extract-all%20metadata.csv%0A%0A%23%20Extract%20metadata%20from%20already%20downloaded%20files%0Apython%20-u%20GotMeta.py%20–extract-only%20–files-dir%20.%2Fdownloads%20–extract-all%20metadata.csv” message=”” highlight=”” provider=”manual”/]

Anti-Bot Protection

Some sites block automated downloads, such as those that use Cloudflare. To prevent this, use a Proxy

1. Open a proxied browser using BurpSuite / ZAP / whatever and browse to the target URL.

2. Manually get through the WAF / Cloudflare / Other Bot protections.

3. Use proxied commands:

   [pastacode lang=”markup” manual=”%23%20Route%20through%20Burp%20Suite%20with%20authenticated%20session%0Apython%20-u%20GotMeta.py%20-d%20example.com%20-k%20YOUR_API_KEY%20-s%20YOUR_SEARCH_ENGINE_ID%20–download%20–extract-all%20metadata.csv%20–proxy%20http%3A%2F%2F127.0.0.1%3A8080″ message=”” highlight=”” provider=”manual”/]