Derby version mismatch between Spark and Hive : Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient -


while connecting spark(1.4.0 built hadoop 2.6.0) hive(version 1.1.0) tables exception coming due derby version mismatch.
there way both(spark & hive) can use same derby version ? ...any other way fix ?

caused by: java.sql.sqlexception: failed start database 'metastore_db' class loader org.apache.spark.sql.hive.client.isolatedclientloader$$anon$1@e0d0b81, see next exception details.     @ org.apache.derby.impl.jdbc.sqlexceptionfactory.getsqlexception(unknown source)     @ org.apache.derby.impl.jdbc.sqlexceptionfactory40.wrapargsfortransportacrossdrda(unknown source)     ... 88 more caused by: java.sql.sqlexception: database @ /home/saurabh/softwares/hive1.1/bin/metastore_db has incompatible format current version of software.  database created or upgraded version 10.11.     @ org.apache.derby.impl.jdbc.sqlexceptionfactory.getsqlexception(unknown source)     @ org.apache.derby.impl.jdbc.sqlexceptionfactory40.wrapargsfortransportacrossdrda(unknown source)     @ org.apache.derby.impl.jdbc.sqlexceptionfactory40.getsqlexception(unknown source)     @ org.apache.derby.impl.jdbc.util.generatecssqlexception(unknown source)     ... 85 more caused by: error xslan: database @ /home/saurabh/softwares/hive1.1/bin/metastore_db has incompatible format current version of software.  database created or upgraded version 10.11.     @ org.apache.derby.iapi.error.standardexception.newexception(unknown source)     @ org.apache.derby.impl.store.raw.log.logtofile.readcontrolfile(unknown source)     @ org.apache.derby.impl.store.raw.log.logtofile.boot(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.boot(unknown source)     @ org.apache.derby.impl.services.monitor.topservice.bootmodule(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.startmodule(unknown source)     @ org.apache.derby.iapi.services.monitor.monitor.bootservicemodule(unknown source)     @ org.apache.derby.impl.store.raw.data.basedatafilefactory.bootlogfactory(unknown source)     @ org.apache.derby.impl.store.raw.data.basedatafilefactory.setrawstorefactory(unknown source)     @ org.apache.derby.impl.store.raw.rawstore.boot(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.boot(unknown source)     @ org.apache.derby.impl.services.monitor.topservice.bootmodule(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.startmodule(unknown source)     @ org.apache.derby.iapi.services.monitor.monitor.bootservicemodule(unknown source)     @ org.apache.derby.impl.store.access.ramaccessmanager.boot(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.boot(unknown source)     @ org.apache.derby.impl.services.monitor.topservice.bootmodule(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.startmodule(unknown source)     @ org.apache.derby.iapi.services.monitor.monitor.bootservicemodule(unknown source)     @ org.apache.derby.impl.db.basicdatabase.bootstore(unknown source)     @ org.apache.derby.impl.db.basicdatabase.boot(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.boot(unknown source)     @ org.apache.derby.impl.services.monitor.topservice.bootmodule(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.bootservice(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.startproviderservice(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.findproviderandstartservice(unknown source)     @ org.apache.derby.impl.services.monitor.basemonitor.startpersistentservice(unknown source)     @ org.apache.derby.iapi.services.monitor.monitor.startpersistentservice(unknown source)     ... 85 more 15/07/01 17:29:49 info sparkui: stopped spark web ui @ http://192.168.17.155:4040 15/07/01 17:29:49 info dagscheduler: stopping dagscheduler

this how solved it:

rm -rf "${appdata//\\//}/../local/temp/spark-"* rm -rf /c/tmp/hive rm -rf ./metastore_db

i deleted git bash the:

  • local spark directories (not mandatory good)
  • the temporary hive directory (mandatory)
  • the metastore_db (it mandatory delete metastore_db/*.loc deleted whole thing sure )

i not sure applicable case, reset derby database can use it.


-

Comments

Post a Comment

Popular posts from this blog

OpenCV OpenCL: Convert Mat to Bitmap in JNI Layer for Android -

there several posts converting mat bitmap using utils.mattobitmap() function. i'm assuming function can called in java layer after importing utils class. i want transfer data memory address pointed uint32_t* bmpcontent; in code below. jniexport void jnicall java_com_nod_nodcv_nodcvactivity_runfilter( jnienv *env, jclass clazz, jobject outbmp, jbytearray indata, jint width, jint height, jint choice, jint filter) { int outsz = width*height; int insz = outsz + outsz/2; androidbitmapinfo bmpinfo; if (androidbitmap_getinfo(env, outbmp, &bmpinfo) < 0) { throwjavaexception(env,"gaussianblur","error retrieving bitmap meta data"); return; } if (bmpinfo.format != android_bitmap_format_rgba_8888) { throwjavaexception(env,"gaussianblur","expecting rgba_8888 format"); return; } uint32_t* bmpcontent; if (androidbitmap_lockpixels(env, outbmp,(void**)&bmpcontent)...
Read more

android - org.xmlpull.v1.XmlPullParserException: expected: START_TAG {http://schemas.xmlsoap.org/soap/envelope/}Envelope -

i trying hit soap api in application login purpose , getting above mentioned error , can 1 tell me going wrong , first time soap apis. here code hit soap :- vector<object> args = new vector<object>(); hashtable<string, object> hashtable = new hashtable<string, object>(); hashtable.put("email", "mss.siddhart28@gmail.com"); hashtable.put("password", "siddharth"); args.add(hashtable); soapserializationenvelope envelope = new soapserializationenvelope( soapenvelope.ver11); envelope.dotnet = true; envelope.xsd = soapserializationenvelope.xsd; envelope.enc = soapserializationenvelope.enc; (new marshalhashtable()).register(envelope); httptransportse androidhttptransport = new httptransportse( "http://tempuri.org/", 15000); androidhttptransport.debug = true; string result = ""; try { soapobject request = new soapobject("http:...
Read more

python - How to remove the Xframe Options header in django? -

i have made page has iframe . inside iframe want show multiple different links article facebook, or news, or youtube video or other possible url. but, due xframe header, unable so. referred following link: https://docs.djangoproject.com/en/1.8/ref/clickjacking/ , django xframeoptionsmiddleware (x-frame-options) - allow iframe client ip but didn't help. my settings.py file's middleware_classes is: middleware_classes = ( 'django.contrib.sessions.middleware.sessionmiddleware', 'django.middleware.common.commonmiddleware', 'django.middleware.csrf.csrfviewmiddleware', 'django.contrib.auth.middleware.authenticationmiddleware', 'django.contrib.auth.middleware.sessionauthenticationmiddleware', 'django.contrib.messages.middleware.messagemiddleware', 'django.middleware.clickjacking.xframeoptionsmiddleware', ) from http://django-secure.readthedocs.org/en/latest/middleware.html , found using ...
Read more