Our issue:
In Hive CLI
INSERT INTO TABLE parquet_table PARTITION(year, month)
SELECT * FROM regular_table;
After a long time, shows the error:
Error in acquireLocks...
FAILED: Error in acquiring locks: Locks on the underlying objects cannot be acquired. retry after some time
Rerun by "hive --hiveconf hive.root.logger=ALL,console"
14/08/11 16:05:37 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 14/08/11 16:05:37 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
14/08/11 16:05:37 INFO ql.Driver: <PERFLOG method=acquireReadWriteLocks>
14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Opening socket connection to server. Will not attempt to authenticate using SASL (java.lang.SecurityException: Unable to locate a login configuration)
14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Socket connection established to , initiating session
14/08/11 16:05:38 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
14/08/11 16:05:38 WARN ZooKeeperHiveLockManager: Possibly transient ZooKeeper exception:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hive_zookeeper_namespace_hive1/default
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.createChild(ZooKeeperHiveLockManager.java:285)
at org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.lockPrimitive(ZooKeeperHiveLockManager.java:354)
at org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.lock(ZooKeeperHiveLockManager.java:303)
at org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.lock(ZooKeeperHiveLockManager.java:220)
at org.apache.hadoop.hive.ql.Driver.acquireReadWriteLocks(Driver.java:858)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:975)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Opening socket connection to server . Will not attempt to authenticate using SASL (java.lang.SecurityException: Unable to locate a login configuration)
14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Socket connection established to initiating session
14/08/11 16:05:38 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 14/08/11 16:05:38 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
14/08/11 16:05:38 DEBUG zookeeper.ClientCnxnSocketNIO: Ignoring exception during shutdown input
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:665)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:430)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:189)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
14/08/11 16:05:38 DEBUG zookeeper.ClientCnxnSocketNIO: Ignoring exception during shutdown output
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:676)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:438)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:196)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
Hive uses ZooKeeper to implement its locking facility, which is then used as a basis of concurrency support, that is, to support concurrent read/write to a same table/partition without causing any corruption to table data/metadata. You can find more details about locking in Hive in following page:
https://cwiki.apache.org/confluence/display/Hive/Locking
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_18_5.html
what is the solution for this?
ReplyDeletesorry I dont understand the error completely . why Locks on the underlying objects cannot be acquired?
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHi ,
ReplyDeletePlease set this property below and run it.
SET hive.support.concurrency=false;
By default it is true.
Regards
Nitish Khanna