Skip to content

About

Crawl accepted papers and citation data from ML/DL/NLP/CV/Robotics/Security/SE conferences

Resources

Stars

64 stars

Watchers

0 watching

Forks

Repository files navigation

paper-explorer

License: MIT Python 3.13+ GitHub stars GitHub last commit Sponsor

Retrieve accepted paper metadata from ML/DL/NLP/CV/Robotics/Security/SE conferences. Uses the OpenReview API, web scraping, CVF Open Access, conf.researchr.org, Crossref, roboticsproceedings.org, and DBLP's SPARQL endpoint.

Getting Started

1. Install uv

uv is a fast Python package manager used to manage dependencies.

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

After installing, restart your terminal so the uv command is available.

2. Clone and install dependencies

git clone https://github.com/brightjade/paper-explorer.git
cd paper-explorer
uv sync
source .venv/bin/activate

This creates a virtual environment and installs all required packages automatically. Activating it once lets you run ppr directly; re-run source .venv/bin/activate in each new shell.

3. Download paper data

The paper data is hosted as a GitHub Release asset. Run the setup script to download and extract it:

./setup.sh

This downloads the latest data snapshot (~95 MB) and extracts it to data/.

Note: Requires either the GitHub CLI (gh) or curl. If you don't have gh, the script falls back to curl automatically.

4. Set up OpenReview credentials (optional)

Only needed if you want to crawl OpenReview conferences (ICLR, NeurIPS, ICML, COLM, CoRL). Other conferences are scraped from public websites and don't require authentication.

  1. Create a free account at https://openreview.net/signup
  2. Copy the example env file and fill in your credentials:
cp .env.example .env
  1. Edit .env with your OpenReview username and password.

Usage

# Crawl one or more conferences
ppr crawl iclr_2025
ppr crawl iclr_2025 neurips_2025 icml_2025

# Enrich with citation counts and abstracts
ppr enrich iclr_2025
ppr enrich iclr_2025 neurips_2025 icml_2025
ppr enrich --all                      # refresh data/; archive years (4+ old) follow a monthly cohort, batch-only

# Every run also re-attempts papers no earlier run could match, and re-checks
# the papers it already matched against the record they are bound to.

# Validate paper counts against DBLP
ppr validate iclr_2025
ppr validate iclr_2025 neurips_2025 --tolerance 0.15

# Check tracked venues for accepted-paper lists we haven't registered yet
ppr discover
ppr discover --venue cvpr --venue iclr   # limit the sweep to some venues
ppr discover --json                      # machine-readable, for the monthly workflow

# Build static JSON for web app
./build.sh

Available conferences

Conference ID format: <venue>_<year> (e.g., iclr_2025). Selections indicate available paper tracks.

ML

Conference 2026 2025 2024 2023
ICLR oral, poster oral, spotlight, poster oral, spotlight, poster oral, spotlight, poster
NeurIPS oral, spotlight, poster oral, spotlight, poster oral, spotlight, poster
ICML main, spotlight oral, spotlight, poster oral, spotlight, poster oral, poster
TMLR main main main main
AAAI main main main main
IJCAI main main main main

NeurIPS also includes datasets_oral, datasets_spotlight, datasets_poster tracks.

NLP

Conference 2026 2025 2024 2023
ACL main, findings, industry main, findings, industry main, findings main, findings, industry
EMNLP main, findings, industry main, findings, industry main, findings, industry
NAACL main, findings, industry main, findings, industry
COLM main main main
EACL main, findings, industry main, findings main, findings
COLING main, industry main

CV (CVF / ECVA)

Conference 2026 2025 2024 2023
CVPR main main main main
ICCV main main
ECCV main
WACV main main main main

Robotics

Conference 2026 2025 2024 2023
CoRL oral, poster main oral, poster
ICRA main main main main
IROS main main main
RSS main main main main

Security

Conference 2026 2025 2024 2023
USENIX Security main main main main

Software Engineering (researchr research tracks)

Conference 2026 2025 2024 2023
ICSE main main main main
FSE main main main main
ASE main main main main
ISSTA main main main main

2013–2022

Earlier years come from DBLP, except where OpenReview keeps the presentation type. DBLP's proceedings volumes separate main, findings and industry papers but not orals.

Venue Years Source Labels
ICLR 2013–2022 DBLP 2013–2016, OpenReview 2017–22 main (2013–2016); oral, poster 2017–19; oral, spotlight, poster 2020–22
NeurIPS 2013–2022 DBLP 2013–2020, OpenReview 2021–22 main (2013–2020); oral, spotlight, poster 2021; main 2022; datasets 2021–22
ICML 2013–2022 DBLP 2013–2022 main
AAAI 2013–2022 DBLP 2013–2022 main
IJCAI 2013, 2015–2022 DBLP 2013, 2015–2022 main
ACL 2013–2022 DBLP 2013–2022 main; findings 2021–2022
EMNLP 2013–2022 DBLP 2013–2022 main; findings 2020–2022; industry 2022
NAACL 2013, 2015–2016, 2018–2019, 2021–2022 DBLP 2013, 2015–2016, 2018–2019, 2021–2022 main; findings 2022; industry 2018–2019, 2021–2022
EACL 2014, 2017, 2021 DBLP 2014, 2017, 2021 main
COLING 2014, 2016, 2018, 2020, 2022 DBLP 2014, 2016, 2018, 2020, 2022 main; industry 2020
CVPR 2013–2022 DBLP 2013–2022 main
ICCV 2013, 2015, 2017, 2019, 2021 DBLP 2013, 2015, 2017, 2019, 2021 main
ECCV 2014, 2016, 2018, 2020, 2022 DBLP 2014, 2016, 2018, 2020, 2022 main
WACV 2013–2022 DBLP 2013–2022 main
CoRL 2017–2022 DBLP 2017–2020, OpenReview 2021–22 main (2017–2020); oral, poster 2021–22
ICRA 2013–2022 DBLP 2013–2022 main
IROS 2013–2022 DBLP 2013–2022 main
RSS 2013–2022 DBLP 2013–2022 main
USENIX Security 2013–2022 DBLP 2013–2022 main
ICSE 2013–2022 DBLP 2013–2022 main
FSE 2013–2022 DBLP 2013–2022 main
ASE 2013–2022 DBLP 2013–2022 main
ISSTA 2013–2022 DBLP 2013–2022 main

Four venue-years differ in scope from 2023 on. ICSE 2013 is one DBLP volume of 242 papers (its research track had about 85), so it also carries other tracks; from 2014 DBLP splits a companion volume off and the backfill excludes it. AAAI's DBLP volumes include student abstracts and demos, which the 2023+ crawl leaves out. FSE and ASE publish every track in one volume, so their 2013–2022 counts are higher than a research-track-only crawl would give.

Literature Survey (Claude Code skill)

Use the /survey command in Claude Code to generate a grounded literature survey from the accepted papers in this repository. Requires enriched data (papers_enriched.jsonl) for the target conferences.

# Specify conferences and year range
/survey I'm exploring efficient inference methods for large language models,
like speculative decoding and early exit strategies. Search NLP and ML
conferences from 2023-2025.

# Let it ask you for scope
/survey What papers exist on 3D object detection from point clouds?

# Target specific venues
/survey Find related work on code generation with LLMs. Search ICSE, FSE,
ASE, ACL, and EMNLP from 2023-2025.

The survey searches through real accepted papers, ranks by citation count, identifies datasets and benchmarks, and highlights research gaps. Output is saved to outputs/<topic>_<timestamp>.md.

About

Crawl accepted papers and citation data from ML/DL/NLP/CV/Robotics/Security/SE conferences

Resources

Stars

64 stars

Watchers

0 watching

Forks

Releases

Sponsor this project

Packages

Contributors

Languages