python - Scrapy: ERROR: Spider error processing -


i new in python & scrapy. tried run existing code, got error on every address:

2015-07-02 01:52:19 [scrapy] debug: crawled (200) <get http://www.tripadvisor.com/showuserreviews-g187147-d197524-r281927613-hotel_mirific_opera-paris_ile_de_france.html> (referer: http://www.tripadvisor.com/hotel_review-g187147-d197524-reviews-hotel_mirific_opera-paris_ile_de_france.html)2015-07-02 01:52:19 [scrapy] error: spider error processing <get http://www.tripadvisor.com/showuserreviews-g187147-d197524-r281927613-hotel_mirific_opera-paris_ile_de_france.html> (referer: http://www.tripadvisor.com/hotel_review-g187147-d197524-reviews-hotel_mirific_opera-paris_ile_de_france.html)

traceback (most recent call last): file "/usr/local/lib/python2.7/dist-packages/scrapy/utils/defer.py", line 102, in iter_errback yield next(it) file "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/offsite.py", line 28, in process_spider_output x in result: file "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/referer.py", line 22, in return (_set_referer(r) r in result or ()) file "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/urllength.py", line 37, in return (r r in result or () if _filter(r)) file "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/depth.py", line 54, in return (r r in result or () if _filter(r)) file "/usr/local/lib/python2.7/dist-packages/scrapy/spiders/crawl.py", line 67, in _parse_response cb_res = callback(response, **cb_kwargs) or () file "/home/talmosko/documents/scrapy/tripadvisor/spiders/tripadvisor.py", line 30, in parse_item item['state'] = hxs.xpath('//*[@id="page"]/div[2]/div[1]/ul/li[2]/a/span/text()').extract()[0].encode('ascii', errors='ignore')

indexerror: list index out of range

this code: http://pastebin.com/xzm5drdd

what problem? seems spide didnt answer..

thanks!

you trying access element doesn't exist, error in line

item['state'] =  hxs.xpath('//*[@id="page"]/div[2]/div[1]/ul/li[2]/a/span/text()').extract()[0].encode('ascii', errors='ignore')

problably 

item['state'] =  hxs.xpath('//*[@id="page"]/div[2]/div[1]/ul/li[2]/a/span/text()').extract()

is empty , trying access first element. have 2 options:


-

Comments

Post a Comment

Popular posts from this blog

OpenCV OpenCL: Convert Mat to Bitmap in JNI Layer for Android -

there several posts converting mat bitmap using utils.mattobitmap() function. i'm assuming function can called in java layer after importing utils class. i want transfer data memory address pointed uint32_t* bmpcontent; in code below. jniexport void jnicall java_com_nod_nodcv_nodcvactivity_runfilter( jnienv *env, jclass clazz, jobject outbmp, jbytearray indata, jint width, jint height, jint choice, jint filter) { int outsz = width*height; int insz = outsz + outsz/2; androidbitmapinfo bmpinfo; if (androidbitmap_getinfo(env, outbmp, &bmpinfo) < 0) { throwjavaexception(env,"gaussianblur","error retrieving bitmap meta data"); return; } if (bmpinfo.format != android_bitmap_format_rgba_8888) { throwjavaexception(env,"gaussianblur","expecting rgba_8888 format"); return; } uint32_t* bmpcontent; if (androidbitmap_lockpixels(env, outbmp,(void**)&bmpcontent)...
Read more

android - org.xmlpull.v1.XmlPullParserException: expected: START_TAG {http://schemas.xmlsoap.org/soap/envelope/}Envelope -

i trying hit soap api in application login purpose , getting above mentioned error , can 1 tell me going wrong , first time soap apis. here code hit soap :- vector<object> args = new vector<object>(); hashtable<string, object> hashtable = new hashtable<string, object>(); hashtable.put("email", "mss.siddhart28@gmail.com"); hashtable.put("password", "siddharth"); args.add(hashtable); soapserializationenvelope envelope = new soapserializationenvelope( soapenvelope.ver11); envelope.dotnet = true; envelope.xsd = soapserializationenvelope.xsd; envelope.enc = soapserializationenvelope.enc; (new marshalhashtable()).register(envelope); httptransportse androidhttptransport = new httptransportse( "http://tempuri.org/", 15000); androidhttptransport.debug = true; string result = ""; try { soapobject request = new soapobject("http:...
Read more

python - How to remove the Xframe Options header in django? -

i have made page has iframe . inside iframe want show multiple different links article facebook, or news, or youtube video or other possible url. but, due xframe header, unable so. referred following link: https://docs.djangoproject.com/en/1.8/ref/clickjacking/ , django xframeoptionsmiddleware (x-frame-options) - allow iframe client ip but didn't help. my settings.py file's middleware_classes is: middleware_classes = ( 'django.contrib.sessions.middleware.sessionmiddleware', 'django.middleware.common.commonmiddleware', 'django.middleware.csrf.csrfviewmiddleware', 'django.contrib.auth.middleware.authenticationmiddleware', 'django.contrib.auth.middleware.sessionauthenticationmiddleware', 'django.contrib.messages.middleware.messagemiddleware', 'django.middleware.clickjacking.xframeoptionsmiddleware', ) from http://django-secure.readthedocs.org/en/latest/middleware.html , found using ...
Read more