python - how to scrape product details on amazon webpage using beautifulsoup -



python - how to scrape product details on amazon webpage using beautifulsoup -

for webpage: http://www.amazon.com/harry-potter-prisoner-azkaban-rowling/dp/0439136369/ref=pd_sim_b_2?ie=utf8&refrid=1mfbraecgpmvzc5mjcwg how scrape product details , output dict in python. in above case, dict output want have be:

age range: 9 - 12 years grade level: 4 - 7 ... ...

i'm new beautifulsoup , didn't find illustration create happen. want have illustration follow.

the thought iterate on product details items help of table#productdetailstable div.content ul li css selector, utilize bold text key , next sibling value:

from pprint import pprint bs4 import beautifulsoup import requests url = 'http://www.amazon.com/dp/0439136369' response = requests.get(url, headers={'user-agent': 'mozilla/5.0 (windows nt 6.2; wow64) applewebkit/537.36 (khtml, gecko) chrome/37.0.2062.120 safari/537.36'}) soup = beautifulsoup(response.content) tags = {} li in soup.select('table#productdetailstable div.content ul li'): try: title = li.b key = title.text.strip().rstrip(':') value = title.next_sibling.strip() tags[key] = value except attributeerror: break pprint(tags)

prints:

{ u'age range': u'9 - 12 years', u'amazon best sellers rank': u'#1,440 in books (', u'average client review': u'', u'grade level': u'4 - 7', u'isbn-10': u'0439136369', u'isbn-13': u'978-0439136365', u'language': u'english', u'lexile measure': u'880l', u'mass market paperback': u'448 pages', u'product dimensions': u'1.2 x 5.2 x 7.8 inches', u'publisher': u'scholastic paperbacks (september 11, 2001)', u'series': u'harry potter (book 3)', u'shipping weight': u'11.2 ounces (' }

note breaking loop nail attributeerror. happens on after there no more bold text within li element.

python web-scraping beautifulsoup scrape

Comments

Post a Comment

Popular posts from this blog

Delphi change the assembly code of a running process -

Delphi change the assembly code of a running process - i'm trying alter address 00741fa5 has push test.009e721c . alter push test.009e71c8 . procedure callback; asm force $9e71c8 end; procedure tform2.btn1click(sender: tobject); var ppid :dword; pprocess : cardinal; begin getwindowthreadprocessid(findwindow(nil,pchar('test')), @ppid); pprocess := openprocess(process_all_access, false, ppid); injectasm(pprocess, $741fa5, 10, integer(@callback), 0); end; here injectasm function (with error checking) function injectasm(process: longword; injectaddress, injectsize, codeaddress, codesize : integer): pointer; var csize, rsize : integer; replaced : array of byte; jmp : array [0..4] of byte; jmpaddress : integer; nopv : byte; : integer; nbr: ulong_ptr; begin //injectsize must equal or greater 5, because need space our //(far) jmp wwxxyyzz instruction //if there's no space, inject in place of few instructions if injectsize ...
Read more

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) -

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) - hello have problem hibernate , jackson. have 2 pojos @entity @table(name = "users") public class user implements serializable { static final long serialversionuid = 4235637833262722256l; @id @generatedvalue private int id; @column(name = "creation_date") private date creationdate; @column(name = "first_name") private string firstname; @column(name = "last_name") private string lastname; @version @column(name = "version") private int version; @column(name = "e_mail") private string email; @column(name = "enabled") private boolean enabled; @column(name = "login") private string login; @column(name = "password") private string password; @jsonmanagedreference("role") @onetomany(mappedby = "user", cascade = cascadetype.all,fetch = fetchtyp...
Read more

C++ 11 "class" keyword -

C++ 11 "class" keyword - i've started using c++ 11 little bit more , have few questions on special uses of class keyword. know it's used declare class, there's 2 instances have seen don't understand: method<class t>(); and class class_name *var; why precede typename keyword class in first example, , keyword pointer in sec example? that known elaborated type specifier , necessary when class name "shadowed" or "hidden" , need explicit. class t { }; // love of god don't t t; t t2; if compiler smart, give these warnings: main.cpp:15:5: error: must utilize 'class' tag refer type 't' in scope t t2; ^ class main.cpp:14:7: note: class 't' hidden non-type declaration of 't' here t t; ^ if class not defined, deed forwards declaration. for example: template <typename t> void method() { using type = typename t::value_type; } method...
Read more