Skip to content
Dai0-2Public

About

SpotAgent: Grounding Visual Geo-localization in Large Vision-Language Models through Agentic Reasoning

Resources

Stars

16 stars

Watchers

0 watching

Forks

Repository files navigation

SpotAgent: Grounding Visual Geo-localization in Large Vision-Language Models through Agentic Reasoning

Research code release for agentic geo-localization: tools + reasoning + verification.

Links

Quick Start

1) Fetch dependencies

bash scripts/setup_deps.sh

1.5) Minimal verification (optional)

bash scripts/verify.sh

2) Create the environment

bash install_spotagent.sh
conda activate spotagent

3) Configure environment variables

cp .env.example .env
set -a
source .env
set +a

Common variables:

  • TAVILY_API_KEY
  • YOU_API_KEY
  • MCP_SERVER_URL
  • GOOGLE_MAPS_API_KEY
  • VLLM_HOST
  • VLLM_PORT
  • MODEL_NAME

4) Launch vLLM

export MODEL_NAME=/path/to/your/model
bash inference/vllm_deploy_qwen25vl.sh

Note: --limit-mm-per-prompt format depends on vLLM version.

  • vLLM 0.13+ expects image=8 (KEY=VALUE)
  • vLLM 0.8.x expects JSON: {"image":8}

You can override via:

export LIMIT_MM_PER_PROMPT='{"image":8}'   # for vLLM 0.8.x
# or
export LIMIT_MM_PER_PROMPT="image=8"       # for vLLM 0.13+

5) Inference

export VLLM_HOST=localhost
export VLLM_PORT=12005
export MCP_SERVER_URL=http://localhost:3000/mcp

python inference/inference_im2gps3k.py \
  --image_dir /path/to/images \
  --output /path/to/output.jsonl \
  --temp_dir /path/to/artifacts \
  --num_workers 4

6) Evaluation

python inference/evaluate.py \
  --pred /path/to/output.jsonl \
  --gt_csv /path/to/gt.csv

Output Example

Inference artifacts and JSONL outputs are organized under .temp/outputs/.

  • Example bundle path: .temp/outputs/agent_check_bundle/
  • Example prediction file: .temp/outputs/agent_check_bundle/agent_check.jsonl
  • Typical structure:
    • <bundle>/<run_name>.jsonl
    • <bundle>/<run_name>_artifacts/

Training Pipeline (SFT → Cold Start → RL)

bash train/sft/run_sft.sh
bash train/cold_start/cold_start_mp16_reason_latlon.sh
bash train/rl/run_mp16_1_5_rw_linear_hard.sh

Training entry points and details are documented in train/README.md.

Project Structure

spotagent/
  inference/
    vllm_deploy_qwen25vl.sh
    inference_im2gps3k.py
    evaluate.py
    utils_vllm.py
    utils_agent_tool.py
    utils.py
  spotagent_tools/
  googlemcp/
  assets/
  scripts/
    setup_deps.sh
  train/

Notes

  • This repo keeps the minimal inference + evaluation stack and excludes large weights and datasets.
  • LLaMA-Factory and verl are external dependencies pulled via scripts/setup_deps.sh.
  • If you need Google Maps geocoding, launch GoogleMCP and set GOOGLE_MAPS_API_KEY.

Customize Paths (important)

Some scripts contain default absolute paths. Please replace them with your local paths when running. Examples can be found in training scripts under train/.

About

SpotAgent: Grounding Visual Geo-localization in Large Vision-Language Models through Agentic Reasoning

Resources

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages