Differences
This shows you the differences between two versions of the page.
|
linux_wiki:amazon_product_page_checker [2014/11/30 22:41] billdozor created |
linux_wiki:amazon_product_page_checker [2019/05/25 23:50] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Amazon Product Page Checker ====== | ||
| - | Goal: Automatically check an Amazon product page for certain information. (Availability, | ||
| - | Prerequisite: | ||
| - | * You can send e-mail from the command line: [[ Send Email via Gmail SMTP Cmd Line ]] | ||
| - | |||
| - | ==== Prep Work: Web Browser ==== | ||
| - | |||
| - | 1) In a web browser, visit the Amazon.com product page you want to auto check and save the URL. | ||
| - | |||
| - | 2) Copy your user agent string. (To trick the web servers later on) | ||
| - | |||
| - | You can find it by visiting: http:// | ||
| - | |||
| - | Alternatively, | ||
| - | * Clicking the Open Menu button (three lines, upper right) | ||
| - | * Click the Developer button, then Network | ||
| - | * Ensure the " | ||
| - | * Click on any of the files that have been successfully loaded | ||
| - | * On the right, a new pane opens up, with the " | ||
| - | * Scroll down, under " | ||
| - | * Select the string to the right of " | ||
| - | |||
| - | ==== Testing Curl and Grep ==== | ||
| - | |||
| - | 1) Using the collected product page URL and user agent string, test a curl against the page. | ||
| - | |||
| - | In this example, I am using a Firefox UA string and checking a PS4 bundle page: | ||
| - | <code bash> | ||
| - | curl -sA " | ||
| - | </ | ||
| - | |||
| - | * The (-s) option makes curl silent and not show download progress, which just gets in the way | ||
| - | * The (-A) option is for specifying user agent string. If you do NOT send a user agent string, Amazon displays an error message in the returned page. | ||
| - | * The last field is the product page URL. | ||
| - | |||
| - | 2) Decide what piece of information from the page you want to watch and test grep. | ||
| - | |||
| - | At the time of the script that I wrote, that PS4 bundle page had a nice big banner on it that said "Sign up to be notified when this item becomes available" | ||
| - | |||
| - | <code bash> | ||
| - | curl -sA " | ||
| - | </ | ||
| - | |||
| - | * Grep's (-o) option, only displays the found text, and nothing else. | ||
| - | * Using this logic, if the item became available to buy, that statement would not return any result. | ||
| - | |||
| - | ==== The Script ==== | ||
| - | |||
| - | Putting the above curl statement into a working script looks like this: | ||
| - | |||
| - | <code bash ps4_checker.sh> | ||
| - | #!/bin/bash | ||
| - | |||
| - | RESPONSE=$(curl -sA " | ||
| - | |||
| - | if [ " | ||
| - | / | ||
| - | | ||
| - | fi | ||
| - | </ | ||
| - | |||
| - | * RESPONSE: a variable that holds the result of the curl. (either the grep search string or nothing) | ||
| - | * if statement : If the grep search string is not on the page, e-mail me and let me know the product is available for purchase! | ||
| - | * / | ||
| - | * crontab -l line: Disable the crontab entry by commenting it out. Since this script is launched at a set interval with cron, once the product is available, I would get an e-mail every so often unless the cron entry was disabled. | ||
| - | * crontab -l : list the crontab entries | ||
| - | * sed / | ||
| - | * s/ | ||
| - | * crontab - : pipe that output line into crontab as input, thus editing the crontab entry | ||
| - | * Do nothing else if the string is still there. | ||
| - | |||
| - | ==== Auto Check it with Cron ==== | ||
| - | Don't forget create the cron entry to have the script execute every 15 mins or so. | ||
| - | |||
| - | <code bash> | ||
| - | crontab -e | ||
| - | </ | ||
| - | |||
| - | <code bash> | ||
| - | # minute (m), hour (h), day of month (dom), month (mon), | ||
| - | # and day of week (dow) or use ' | ||
| - | # | ||
| - | # m h dom mon dow | ||
| - | */15 * * * * / | ||
| - | </ | ||
