rust-scraper1

Rust - Reqwest and Scraper example

This is an example web scraper project. It downloads and extracts the Bible from https://www.bible.com.

User-Agent

The program will update the user-agent string it uses via https://www.mozilla.org/en-US/firefox/releases/ because I can.

-a or --agent-update

Bible versions

• ESV
• NIV
• YLT98
• KJV
• NASB2020
• MKJV

-v NIV

Relative/absolute URLs

See relative_to_absolute(url: &str, href: &str) -> Result to properly handle those.

Using

• Copy initial.config to app.config.
  
• Make a bible directory.
• Run the fetch command.
• Run the extract command. Try the --output option.
• Run the verse command.