code.ivysaur.me/nyaascrape
Archived
1
0
Fork 0
Code Issues Pull Requests Releases Activity
A scraper and ui for maintaining a mirror of NyaaTorrents. https://code.ivysaur.me/nyaascrape
This repository has been archived on 2020-05-03. You can view files and clone it, but cannot push or open issues or pull requests.
2 Commits 1 Branch 0 Tags 228 KiB
master
Go to file
mappu ba5ca200c7 commit all archived files
2014-10-05 00:00:00 +00:00
dist-archive commit all archived files 2014-10-05 00:00:00 +00:00
doc commit all archived files 2014-10-05 00:00:00 +00:00
.legacy-codesite.toml initial meta commit 2014-10-04 00:00:00 +00:00
README.md commit all archived files 2014-10-05 00:00:00 +00:00

README.md

nyaascrape

a scraper and ui for maintaining a mirror of NyaaTorrents.

Tags: anime, torrent, scraper

Scraper

The command-line tool can be used to build up an sqlite3 database of most information hosted on Nyaa. It updates an existing sqlite3 database or creates a new one. To bring your database up-to-date, the --update option will determine the most recent torrents available and set the download-chunk options appropriately. It would be suitable to run this via cron.

Usage: nyaascrape [options]
Options:
  -b, --batches {number}         Number of batches to download
  -c, --count {number}           Number of torrents to download per batch
  --database {filename}          Select output file
  --get-latest-id                Find the most recent Torrent ID on site
  --help                         Display this message
  --no-torrents                  Download entries only, no torrent files
  -s, --start {number}           Torrent ID to start processing from
  --sukebei                      Download from sukebei instead of www
  --test-file {file}             Test parse local file containing nyaa html
  -q                             Quiet, suppress output
  --update                       Automatically set -s/-c/-b options to get all
                                   the latest torrent entries
  --url {http://.../}            Set custom nyaa URL

Web Interface

The web interface allows browse, search, and torrent downloading using an interface modelled on the live NyaaTorrents site. However, the following issues make it a little unsuitable for public / high-traffic access:

  • It shamelessly uses nyaa's own images and layout
  • It works directly from the unindexed sqlite DB, so searches may be very slow against a full dump

Changelog

2014-10-05: r13

  • Feature: Automatically set range parameters to bring database up-to-date (--update option)
  • Feature: Set custom nyaa URL (--url option)
  • Feature: Display most recent ID on site (--get-latest-id option)
  • Sort usage-help alphabetically
  • Remove site statistics from usage-help display
  • ⬇️ nyaascrape_r13.7z (66.28 KiB)

2014-10-04: r6

See Also