Symptom:
Reading table blocks creating tables in the same database when hive.support.concurrency=true.Creating table will fail with below errors due to database level lock.
hive> create table test(id int); conflicting lock present for default mode EXCLUSIVE
Env:
Hive 0.13Root Cause:
By default in Hive 0.13 when hive.support.concurrency=true, it is usinghive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager hive.lock.manager=org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManagerSome DDLs needs Exclusive lock on database using this transaction manager.
This issue is reported in HIVE-9199 which is fixed in Hive 1.1.
Minimum reproduce:
1. Set hive.support.concurrency=true in hive-site.xml.2. Open 2 hive sessions:
--Session A: create table passwords (col0 string, col1 string, col2 string, col3 string, col4 string, col5 string, col6 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ":"; load data local inpath "/etc/passwd" into table passwords; select count(*) from passwords a,passwords b, passwords c, passwords d where a.col0=b.col0 and a.col0=c.col0 and a.col0=d.col0; --Session B: hive> create table test(id int); conflicting lock present for default mode EXCLUSIVE
Solution:
This is current lock behavior in Hive 0.13. You can disable this feature by setting hive.support.concurrency=false in hive-site.xml.Or upgrade to Hive 0.14 or above in which new lock manager DbLockManager and new transaction manager DbTxnManager are introduced.
No comments:
Post a Comment