code.ivysaur.me
/
nyaascrape Archived
1
0
Fork 0
Code Issues 0 Pull Requests 0 Releases 0 Activity
A scraper and ui for maintaining a mirror of NyaaTorrents. https://code.ivysaur.me/nyaascrape
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
This repo is archived. You can view files and clone it, but cannot push or open issues/pull-requests.
2 Commits
1 Branch
289 KiB
Branch: master
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'master'
${ noResults }
 ZIP  TAR.GZ
mappu ba5ca200c7 commit all archived files 6 years ago
dist-archive commit all archived files 6 years ago
doc commit all archived files 6 years ago
.legacy-codesite.toml initial meta commit 6 years ago
README.md commit all archived files 6 years ago

README.md

nyaascrape

a scraper and ui for maintaining a mirror of NyaaTorrents.

Tags: anime, torrent, scraper

Scraper

The command-line tool can be used to build up an sqlite3 database of most information hosted on Nyaa. It updates an existing sqlite3 database or creates a new one. To bring your database up-to-date, the --update option will determine the most recent torrents available and set the download-chunk options appropriately. It would be suitable to run this via cron.

Usage: nyaascrape [options]
Options:
  -b, --batches {number}         Number of batches to download
  -c, --count {number}           Number of torrents to download per batch
  --database {filename}          Select output file
  --get-latest-id                Find the most recent Torrent ID on site
  --help                         Display this message
  --no-torrents                  Download entries only, no torrent files
  -s, --start {number}           Torrent ID to start processing from
  --sukebei                      Download from sukebei instead of www
  --test-file {file}             Test parse local file containing nyaa html
  -q                             Quiet, suppress output
  --update                       Automatically set -s/-c/-b options to get all
                                   the latest torrent entries
  --url {http://.../}            Set custom nyaa URL

Web Interface

The web interface allows browse, search, and torrent downloading using an interface modelled on the live NyaaTorrents site. However, the following issues make it a little unsuitable for public / high-traffic access:

  • It shamelessly uses nyaa's own images and layout
  • It works directly from the unindexed sqlite DB, so searches may be very slow against a full dump

Changelog

2014-10-05: r13

  • Feature: Automatically set range parameters to bring database up-to-date (--update option)
  • Feature: Set custom nyaa URL (--url option)
  • Feature: Display most recent ID on site (--get-latest-id option)
  • Sort usage-help alphabetically
  • Remove site statistics from usage-help display
  • ⬇️ nyaascrape_r13.7z (66.28 KiB)

2014-10-04: r6

See Also