org.scalatest.exceptions.TestFailedException: spark-submit returned with exit code 1. Command line: './bin/spark-submit' '--name' 'prepare testing tables' '--master' 'local[2]' '--conf' 'spark.ui.enabled=false' '--conf' 'spark.master.rest.enabled=false' '--conf' 'spark.sql.warehouse.dir=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8' '--conf' 'spark.sql.test.version.index=2' '--driver-java-options' '-Dderby.system.home=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8' '/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py' 2018-09-21 04:02:49.264 - stdout> 2018-09-21 04:02:49 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-09-21 04:02:50.054 - stdout> 2018-09-21 04:02:50 INFO SparkContext:54 - Running Spark version 2.3.1 2018-09-21 04:02:50.08 - stdout> 2018-09-21 04:02:50 INFO SparkContext:54 - Submitted application: prepare testing tables 2018-09-21 04:02:50.145 - stdout> 2018-09-21 04:02:50 INFO SecurityManager:54 - Changing view acls to: jenkins 2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO SecurityManager:54 - Changing modify acls to: jenkins 2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO SecurityManager:54 - Changing view acls groups to: 2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO SecurityManager:54 - Changing modify acls groups to: 2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jenkins); groups with view permissions: Set(); users with modify permissions: Set(jenkins); groups with modify permissions: Set() 2018-09-21 04:02:50.469 - stdout> 2018-09-21 04:02:50 INFO Utils:54 - Successfully started service 'sparkDriver' on port 35970. 2018-09-21 04:02:50.493 - stdout> 2018-09-21 04:02:50 INFO SparkEnv:54 - Registering MapOutputTracker 2018-09-21 04:02:50.513 - stdout> 2018-09-21 04:02:50 INFO SparkEnv:54 - Registering BlockManagerMaster 2018-09-21 04:02:50.515 - stdout> 2018-09-21 04:02:50 INFO BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 2018-09-21 04:02:50.516 - stdout> 2018-09-21 04:02:50 INFO BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up 2018-09-21 04:02:50.525 - stdout> 2018-09-21 04:02:50 INFO DiskBlockManager:54 - Created local directory at /tmp/blockmgr-082a5fe3-a63c-4ebf-8302-997c2f117f07 2018-09-21 04:02:50.544 - stdout> 2018-09-21 04:02:50 INFO MemoryStore:54 - MemoryStore started with capacity 366.3 MB 2018-09-21 04:02:50.559 - stdout> 2018-09-21 04:02:50 INFO SparkEnv:54 - Registering OutputCommitCoordinator 2018-09-21 04:02:50.819 - stdout> 2018-09-21 04:02:50 INFO SparkContext:54 - Added file file:/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py at file:/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py with timestamp 1537527770819 2018-09-21 04:02:50.821 - stdout> 2018-09-21 04:02:50 INFO Utils:54 - Copying /home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py to /tmp/spark-adcf2ed2-8598-4f16-af07-1b57ca0f11ad/userFiles-f883abbc-d37f-41a5-98c6-897d51808f9b/test5234199047625563812.py 2018-09-21 04:02:50.89 - stdout> 2018-09-21 04:02:50 INFO Executor:54 - Starting executor ID driver on host localhost 2018-09-21 04:02:50.909 - stdout> 2018-09-21 04:02:50 INFO Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 39797. 2018-09-21 04:02:50.91 - stdout> 2018-09-21 04:02:50 INFO NettyBlockTransferService:54 - Server created on amp-jenkins-worker-02.amp:39797 2018-09-21 04:02:50.911 - stdout> 2018-09-21 04:02:50 INFO BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 2018-09-21 04:02:50.942 - stdout> 2018-09-21 04:02:50 INFO BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None) 2018-09-21 04:02:50.946 - stdout> 2018-09-21 04:02:50 INFO BlockManagerMasterEndpoint:54 - Registering block manager amp-jenkins-worker-02.amp:39797 with 366.3 MB RAM, BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None) 2018-09-21 04:02:50.949 - stdout> 2018-09-21 04:02:50 INFO BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None) 2018-09-21 04:02:50.949 - stdout> 2018-09-21 04:02:50 INFO BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None) 2018-09-21 04:02:51.117 - stdout> 2018-09-21 04:02:51 INFO log:192 - Logging initialized @3329ms 2018-09-21 04:02:51.262 - stdout> 2018-09-21 04:02:51 INFO SharedState:54 - Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8'). 2018-09-21 04:02:51.262 - stdout> 2018-09-21 04:02:51 INFO SharedState:54 - Warehouse path is '/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8'. 2018-09-21 04:02:51.712 - stdout> 2018-09-21 04:02:51 INFO StateStoreCoordinatorRef:54 - Registered StateStoreCoordinator endpoint 2018-09-21 04:02:52.135 - stdout> 2018-09-21 04:02:52 INFO HiveUtils:54 - Initializing HiveMetastoreConnection version 1.2.1 using Spark classes. 2018-09-21 04:02:52.8 - stdout> 2018-09-21 04:02:52 INFO HiveMetaStore:589 - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2018-09-21 04:02:52.827 - stdout> 2018-09-21 04:02:52 INFO ObjectStore:289 - ObjectStore, initialize called 2018-09-21 04:02:52.964 - stdout> 2018-09-21 04:02:52 INFO Persistence:77 - Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 2018-09-21 04:02:52.964 - stdout> 2018-09-21 04:02:52 INFO Persistence:77 - Property datanucleus.cache.level2 unknown - will be ignored 2018-09-21 04:02:56.008 - stdout> 2018-09-21 04:02:56 INFO ObjectStore:370 - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 2018-09-21 04:02:56.11 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.115 - stdout> 2018-09-21 04:02:56 WARN MetaStoreDirectSql:188 - Database initialization failed; direct SQL is disabled 2018-09-21 04:02:56.115 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.115 - stdout> at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600) 2018-09-21 04:02:56.115 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 2018-09-21 04:02:56.115 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.116 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.116 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.116 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.116 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.116 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74) 2018-09-21 04:02:56.117 - stdout> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641) 2018-09-21 04:02:56.117 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.117 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.117 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.117 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.117 - stdout> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 2018-09-21 04:02:56.117 - stdout> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 2018-09-21 04:02:56.117 - stdout> at py4j.Gateway.invoke(Gateway.java:282) 2018-09-21 04:02:56.117 - stdout> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 2018-09-21 04:02:56.117 - stdout> at py4j.commands.CallCommand.execute(CallCommand.java:79) 2018-09-21 04:02:56.117 - stdout> at py4j.GatewayConnection.run(GatewayConnection.java:238) 2018-09-21 04:02:56.117 - stdout> at java.lang.Thread.run(Thread.java:745) 2018-09-21 04:02:56.117 - stdout> NestedThrowablesStackTrace: 2018-09-21 04:02:56.117 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.117 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:960) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:952) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:129) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:481) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.store.query.Query.executeQuery(Query.java:1744) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.store.query.Query.execute(Query.java:1654) 2018-09-21 04:02:56.117 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) 2018-09-21 04:02:56.117 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.118 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.118 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.118 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74) 2018-09-21 04:02:56.118 - stdout> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.118 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.118 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.118 - stdout> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 2018-09-21 04:02:56.119 - stdout> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 2018-09-21 04:02:56.119 - stdout> at py4j.Gateway.invoke(Gateway.java:282) 2018-09-21 04:02:56.119 - stdout> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 2018-09-21 04:02:56.119 - stdout> at py4j.commands.CallCommand.execute(CallCommand.java:79) 2018-09-21 04:02:56.119 - stdout> at py4j.GatewayConnection.run(GatewayConnection.java:238) 2018-09-21 04:02:56.119 - stdout> at java.lang.Thread.run(Thread.java:745) 2018-09-21 04:02:56.119 - stdout> 2018-09-21 04:02:56 INFO ObjectStore:272 - Initialized ObjectStore 2018-09-21 04:02:56.312 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.314 - stdout> 2018-09-21 04:02:56 WARN HiveMetaStore:622 - Retrying creating default database after error: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.315 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.315 - stdout> at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600) 2018-09-21 04:02:56.315 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.315 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114) 2018-09-21 04:02:56.315 - stdout> at com.sun.proxy.$Proxy14.verifySchema(Unknown Source) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.315 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.315 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.315 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.315 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.315 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74) 2018-09-21 04:02:56.316 - stdout> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.316 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.316 - stdout> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 2018-09-21 04:02:56.316 - stdout> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 2018-09-21 04:02:56.316 - stdout> at py4j.Gateway.invoke(Gateway.java:282) 2018-09-21 04:02:56.316 - stdout> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 2018-09-21 04:02:56.316 - stdout> at py4j.commands.CallCommand.execute(CallCommand.java:79) 2018-09-21 04:02:56.316 - stdout> at py4j.GatewayConnection.run(GatewayConnection.java:238) 2018-09-21 04:02:56.316 - stdout> at java.lang.Thread.run(Thread.java:745) 2018-09-21 04:02:56.316 - stdout> NestedThrowablesStackTrace: 2018-09-21 04:02:56.316 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.316 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.ExecutionContextImpl.hasPersistenceInformationForClass(ExecutionContextImpl.java:5768) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:258) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.store.query.Query.executeQuery(Query.java:1744) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.store.query.Query.execute(Query.java:1654) 2018-09-21 04:02:56.316 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.316 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114) 2018-09-21 04:02:56.316 - stdout> at com.sun.proxy.$Proxy14.verifySchema(Unknown Source) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.316 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.316 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.317 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.317 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.317 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74) 2018-09-21 04:02:56.317 - stdout> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.317 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.317 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.317 - stdout> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 2018-09-21 04:02:56.317 - stdout> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 2018-09-21 04:02:56.317 - stdout> at py4j.Gateway.invoke(Gateway.java:282) 2018-09-21 04:02:56.317 - stdout> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 2018-09-21 04:02:56.317 - stdout> at py4j.commands.CallCommand.execute(CallCommand.java:79) 2018-09-21 04:02:56.317 - stdout> at py4j.GatewayConnection.run(GatewayConnection.java:238) 2018-09-21 04:02:56.317 - stdout> at java.lang.Thread.run(Thread.java:745) 2018-09-21 04:02:56.317 - stdout> 2018-09-21 04:02:56 INFO HiveMetaStore:589 - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2018-09-21 04:02:56.317 - stdout> 2018-09-21 04:02:56 INFO ObjectStore:289 - ObjectStore, initialize called 2018-09-21 04:02:56.326 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.328 - stdout> 2018-09-21 04:02:56 WARN MetaStoreDirectSql:188 - Database initialization failed; direct SQL is disabled 2018-09-21 04:02:56.329 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.329 - stdout> at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600) 2018-09-21 04:02:56.329 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.329 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.329 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.329 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.329 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195) 2018-09-21 04:02:56.329 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74) 2018-09-21 04:02:56.33 - stdout> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641) 2018-09-21 04:02:56.33 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2018-09-21 04:02:56.33 - stdout> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2018-09-21 04:02:56.33 - stdout> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2018-09-21 04:02:56.33 - stdout> at java.lang.reflect.Method.invoke(Method.java:497) 2018-09-21 04:02:56.33 - stdout> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 2018-09-21 04:02:56.33 - stdout> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 2018-09-21 04:02:56.33 - stdout> at py4j.Gateway.invoke(Gateway.java:282) 2018-09-21 04:02:56.33 - stdout> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 2018-09-21 04:02:56.33 - stdout> at py4j.commands.CallCommand.execute(CallCommand.java:79) 2018-09-21 04:02:56.33 - stdout> at py4j.GatewayConnection.run(GatewayConnection.java:238) 2018-09-21 04:02:56.33 - stdout> at java.lang.Thread.run(Thread.java:745) 2018-09-21 04:02:56.33 - stdout> NestedThrowablesStackTrace: 2018-09-21 04:02:56.33 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.33 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo" 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:960) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:952) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:129) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:481) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.store.query.Query.executeQuery(Query.java:1744) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.store.query.Query.execute(Query.java:1654) 2018-09-21 04:02:56.33 - stdout> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) 2018-09-21 04:02:56.33 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.331 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) 2018-09-21 04:02:56.331 - stdout> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 2018-09-21 04:02:56.331 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180) 2018-09-21 04:02:56.331 - stdout> at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018-09-21 04:02:56.331 - stdout> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018-09-21 04:02:56.331 - stdout> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 2018-09-21 04:02:56.331 - stdout> at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264) 2018-09-21 04:02:56.331 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385) 2018-09-21 04:02:56.331 - stdout> at org.apache.spark.sql.hive.HiveUtils$.newClientF

sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: spark-submit returned with exit code 1.
Command line: './bin/spark-submit' '--name' 'prepare testing tables' '--master' 'local[2]' '--conf' 'spark.ui.enabled=false' '--conf' 'spark.master.rest.enabled=false' '--conf' 'spark.sql.warehouse.dir=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8' '--conf' 'spark.sql.test.version.index=2' '--driver-java-options' '-Dderby.system.home=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8' '/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py'

2018-09-21 04:02:49.264 - stdout> 2018-09-21 04:02:49 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-09-21 04:02:50.054 - stdout> 2018-09-21 04:02:50 INFO  SparkContext:54 - Running Spark version 2.3.1
2018-09-21 04:02:50.08 - stdout> 2018-09-21 04:02:50 INFO  SparkContext:54 - Submitted application: prepare testing tables
2018-09-21 04:02:50.145 - stdout> 2018-09-21 04:02:50 INFO  SecurityManager:54 - Changing view acls to: jenkins
2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO  SecurityManager:54 - Changing modify acls to: jenkins
2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO  SecurityManager:54 - Changing view acls groups to: 
2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO  SecurityManager:54 - Changing modify acls groups to: 
2018-09-21 04:02:50.146 - stdout> 2018-09-21 04:02:50 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(jenkins); groups with view permissions: Set(); users  with modify permissions: Set(jenkins); groups with modify permissions: Set()
2018-09-21 04:02:50.469 - stdout> 2018-09-21 04:02:50 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 35970.
2018-09-21 04:02:50.493 - stdout> 2018-09-21 04:02:50 INFO  SparkEnv:54 - Registering MapOutputTracker
2018-09-21 04:02:50.513 - stdout> 2018-09-21 04:02:50 INFO  SparkEnv:54 - Registering BlockManagerMaster
2018-09-21 04:02:50.515 - stdout> 2018-09-21 04:02:50 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2018-09-21 04:02:50.516 - stdout> 2018-09-21 04:02:50 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up
2018-09-21 04:02:50.525 - stdout> 2018-09-21 04:02:50 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-082a5fe3-a63c-4ebf-8302-997c2f117f07
2018-09-21 04:02:50.544 - stdout> 2018-09-21 04:02:50 INFO  MemoryStore:54 - MemoryStore started with capacity 366.3 MB
2018-09-21 04:02:50.559 - stdout> 2018-09-21 04:02:50 INFO  SparkEnv:54 - Registering OutputCommitCoordinator
2018-09-21 04:02:50.819 - stdout> 2018-09-21 04:02:50 INFO  SparkContext:54 - Added file file:/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py at file:/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py with timestamp 1537527770819
2018-09-21 04:02:50.821 - stdout> 2018-09-21 04:02:50 INFO  Utils:54 - Copying /home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/test5234199047625563812.py to /tmp/spark-adcf2ed2-8598-4f16-af07-1b57ca0f11ad/userFiles-f883abbc-d37f-41a5-98c6-897d51808f9b/test5234199047625563812.py
2018-09-21 04:02:50.89 - stdout> 2018-09-21 04:02:50 INFO  Executor:54 - Starting executor ID driver on host localhost
2018-09-21 04:02:50.909 - stdout> 2018-09-21 04:02:50 INFO  Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 39797.
2018-09-21 04:02:50.91 - stdout> 2018-09-21 04:02:50 INFO  NettyBlockTransferService:54 - Server created on amp-jenkins-worker-02.amp:39797
2018-09-21 04:02:50.911 - stdout> 2018-09-21 04:02:50 INFO  BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
2018-09-21 04:02:50.942 - stdout> 2018-09-21 04:02:50 INFO  BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None)
2018-09-21 04:02:50.946 - stdout> 2018-09-21 04:02:50 INFO  BlockManagerMasterEndpoint:54 - Registering block manager amp-jenkins-worker-02.amp:39797 with 366.3 MB RAM, BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None)
2018-09-21 04:02:50.949 - stdout> 2018-09-21 04:02:50 INFO  BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None)
2018-09-21 04:02:50.949 - stdout> 2018-09-21 04:02:50 INFO  BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, amp-jenkins-worker-02.amp, 39797, None)
2018-09-21 04:02:51.117 - stdout> 2018-09-21 04:02:51 INFO  log:192 - Logging initialized @3329ms
2018-09-21 04:02:51.262 - stdout> 2018-09-21 04:02:51 INFO  SharedState:54 - Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8').
2018-09-21 04:02:51.262 - stdout> 2018-09-21 04:02:51 INFO  SharedState:54 - Warehouse path is '/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.6/target/tmp/warehouse-0b65366d-3220-4b77-9441-a1a6109406e8'.
2018-09-21 04:02:51.712 - stdout> 2018-09-21 04:02:51 INFO  StateStoreCoordinatorRef:54 - Registered StateStoreCoordinator endpoint
2018-09-21 04:02:52.135 - stdout> 2018-09-21 04:02:52 INFO  HiveUtils:54 - Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
2018-09-21 04:02:52.8 - stdout> 2018-09-21 04:02:52 INFO  HiveMetaStore:589 - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2018-09-21 04:02:52.827 - stdout> 2018-09-21 04:02:52 INFO  ObjectStore:289 - ObjectStore, initialize called
2018-09-21 04:02:52.964 - stdout> 2018-09-21 04:02:52 INFO  Persistence:77 - Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2018-09-21 04:02:52.964 - stdout> 2018-09-21 04:02:52 INFO  Persistence:77 - Property datanucleus.cache.level2 unknown - will be ignored
2018-09-21 04:02:56.008 - stdout> 2018-09-21 04:02:56 INFO  ObjectStore:370 - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2018-09-21 04:02:56.11 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.115 - stdout> 2018-09-21 04:02:56 WARN  MetaStoreDirectSql:188 - Database initialization failed; direct SQL is disabled
2018-09-21 04:02:56.115 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.115 - stdout> 	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600)
2018-09-21 04:02:56.115 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
2018-09-21 04:02:56.115 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.116 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.116 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.116 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.116 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
2018-09-21 04:02:56.117 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.117 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.117 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.117 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.117 - stdout> 	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2018-09-21 04:02:56.117 - stdout> 	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2018-09-21 04:02:56.117 - stdout> 	at py4j.Gateway.invoke(Gateway.java:282)
2018-09-21 04:02:56.117 - stdout> 	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2018-09-21 04:02:56.117 - stdout> 	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2018-09-21 04:02:56.117 - stdout> 	at py4j.GatewayConnection.run(GatewayConnection.java:238)
2018-09-21 04:02:56.117 - stdout> 	at java.lang.Thread.run(Thread.java:745)
2018-09-21 04:02:56.117 - stdout> NestedThrowablesStackTrace:
2018-09-21 04:02:56.117 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.117 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:960)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:952)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:129)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:481)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.store.query.Query.executeQuery(Query.java:1744)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.store.query.Query.execute(Query.java:1654)
2018-09-21 04:02:56.117 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
2018-09-21 04:02:56.117 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.118 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.118 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
2018-09-21 04:02:56.118 - stdout> 	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.118 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.118 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.118 - stdout> 	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2018-09-21 04:02:56.119 - stdout> 	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2018-09-21 04:02:56.119 - stdout> 	at py4j.Gateway.invoke(Gateway.java:282)
2018-09-21 04:02:56.119 - stdout> 	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2018-09-21 04:02:56.119 - stdout> 	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2018-09-21 04:02:56.119 - stdout> 	at py4j.GatewayConnection.run(GatewayConnection.java:238)
2018-09-21 04:02:56.119 - stdout> 	at java.lang.Thread.run(Thread.java:745)
2018-09-21 04:02:56.119 - stdout> 2018-09-21 04:02:56 INFO  ObjectStore:272 - Initialized ObjectStore
2018-09-21 04:02:56.312 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.314 - stdout> 2018-09-21 04:02:56 WARN  HiveMetaStore:622 - Retrying creating default database after error: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.315 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.315 - stdout> 	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600)
2018-09-21 04:02:56.315 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.315 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
2018-09-21 04:02:56.315 - stdout> 	at com.sun.proxy.$Proxy14.verifySchema(Unknown Source)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.315 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.315 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.315 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.315 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.316 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.316 - stdout> 	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2018-09-21 04:02:56.316 - stdout> 	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2018-09-21 04:02:56.316 - stdout> 	at py4j.Gateway.invoke(Gateway.java:282)
2018-09-21 04:02:56.316 - stdout> 	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2018-09-21 04:02:56.316 - stdout> 	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2018-09-21 04:02:56.316 - stdout> 	at py4j.GatewayConnection.run(GatewayConnection.java:238)
2018-09-21 04:02:56.316 - stdout> 	at java.lang.Thread.run(Thread.java:745)
2018-09-21 04:02:56.316 - stdout> NestedThrowablesStackTrace:
2018-09-21 04:02:56.316 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.316 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.ExecutionContextImpl.hasPersistenceInformationForClass(ExecutionContextImpl.java:5768)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:258)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.store.query.Query.executeQuery(Query.java:1744)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.store.query.Query.execute(Query.java:1654)
2018-09-21 04:02:56.316 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6721)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.316 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
2018-09-21 04:02:56.316 - stdout> 	at com.sun.proxy.$Proxy14.verifySchema(Unknown Source)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.316 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.316 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.317 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.317 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
2018-09-21 04:02:56.317 - stdout> 	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.317 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.317 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.317 - stdout> 	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2018-09-21 04:02:56.317 - stdout> 	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2018-09-21 04:02:56.317 - stdout> 	at py4j.Gateway.invoke(Gateway.java:282)
2018-09-21 04:02:56.317 - stdout> 	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2018-09-21 04:02:56.317 - stdout> 	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2018-09-21 04:02:56.317 - stdout> 	at py4j.GatewayConnection.run(GatewayConnection.java:238)
2018-09-21 04:02:56.317 - stdout> 	at java.lang.Thread.run(Thread.java:745)
2018-09-21 04:02:56.317 - stdout> 2018-09-21 04:02:56 INFO  HiveMetaStore:589 - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2018-09-21 04:02:56.317 - stdout> 2018-09-21 04:02:56 INFO  ObjectStore:289 - ObjectStore, initialize called
2018-09-21 04:02:56.326 - stdout> 2018-09-21 04:02:56 ERROR MetaData:115 - Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.328 - stdout> 2018-09-21 04:02:56 WARN  MetaStoreDirectSql:188 - Database initialization failed; direct SQL is disabled
2018-09-21 04:02:56.329 - stdout> javax.jdo.JDOException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.329 - stdout> 	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:600)
2018-09-21 04:02:56.329 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.329 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.329 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.329 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:287)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
2018-09-21 04:02:56.329 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
2018-09-21 04:02:56.33 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2018-09-21 04:02:56.33 - stdout> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2018-09-21 04:02:56.33 - stdout> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-21 04:02:56.33 - stdout> 	at java.lang.reflect.Method.invoke(Method.java:497)
2018-09-21 04:02:56.33 - stdout> 	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2018-09-21 04:02:56.33 - stdout> 	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2018-09-21 04:02:56.33 - stdout> 	at py4j.Gateway.invoke(Gateway.java:282)
2018-09-21 04:02:56.33 - stdout> 	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2018-09-21 04:02:56.33 - stdout> 	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2018-09-21 04:02:56.33 - stdout> 	at py4j.GatewayConnection.run(GatewayConnection.java:238)
2018-09-21 04:02:56.33 - stdout> 	at java.lang.Thread.run(Thread.java:745)
2018-09-21 04:02:56.33 - stdout> NestedThrowablesStackTrace:
2018-09-21 04:02:56.33 - stdout> Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.33 - stdout> org.datanucleus.exceptions.NucleusException: Error opening the Meta-Data file "jar:file:/tmp/test-spark/spark-2.3.1/jars/hive-metastore-1.2.1.spark2.jar!/package.jdo"
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.metadata.xml.MetaDataParser.parseMetaDataURL(MetaDataParser.java:145)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.parseFile(JDOMetaDataManager.java:240)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:773)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.api.jdo.metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:383)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.metadata.MetaDataManager.getMetaDataForClass(MetaDataManager.java:1570)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:960)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.getType(JavaQueryCompiler.java:952)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.expression.PrimaryExpression.bind(PrimaryExpression.java:129)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:481)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.store.query.Query.executeQuery(Query.java:1744)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.store.query.Query.execute(Query.java:1654)
2018-09-21 04:02:56.33 - stdout> 	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:183)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:137)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:295)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
2018-09-21 04:02:56.33 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.331 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:180)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:114)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018-09-21 04:02:56.331 - stdout> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018-09-21 04:02:56.331 - stdout> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:385)
2018-09-21 04:02:56.331 - stdout> 	at org.apache.spark.sql.hi