Tuesday, February 10, 2015

MapR DB cheat sheet

1. Configure max row size.
maprcli config save -values {"mfs.db.max.rowsize.kb":<value in KB>}
maprcli config load -json | grep mfs.db.max.rowsize.kb
2. Table mapping
core-site.xml:

<property>
<name>hbase.table.namespace.mappings</name>
<value>*:/tables</value>
</property>
3. List recent accessed tables.
maprcli table listrecent
4. Create table
a. hbase shell
create '/mapr/demo.mapr.com/tables/tbl1', 'stage'
b. maprcli table
maprcli table create -path /mapr/demo.mapr.com/tables/tbl2
5. Add column family
TBD 
6. Table Info including table size
maprcli table info -path /table
7. Change table permission
maprcli table cf edit -path ${tblPath} -cfname ${colFam} -readperm "u:mapr|u:root" 
8. Check and set max row size for MapR Table.
maprcli config load -json | grep rowsize
            "mfs.db.max.rowsize.kb":"16384",
maprcli config save -values {"mfs.db.max.rowsize.kb":<value in KB>}
9. Compression validation test on every node.
hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://my.cluster.com/tmp/passwd snappy
10. List region list
#  maprcli table region list -path /maprtable
numberofrows  fid              secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
1237798       2192.32.131254   yarn-94         yarn-92      0                       -INFINITY      56393728     0              \x00\x12\xE3'  56762368
2480060       2114.147.262606  yarn-92         yarn-94      0                       \x00\x12\xE3'  113352704    0              \x008\xBA\xE3  114089984
6282142       2193.32.131216   yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
11. Manul Region Split
#  maprcli table region list -path /maprtable
numberofrows  fid              secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
1237798       2192.32.131254   yarn-94         yarn-92      0                       -INFINITY      56393728     0              \x00\x12\xE3'  56762368
2480060       2114.147.262606  yarn-92         yarn-94      0                       \x00\x12\xE3'  113352704    0              \x008\xBA\xE3  114089984
6282142       2193.32.131216   yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
#  maprcli table region split -path /maprtable -fid 2114.147.262606
#  maprcli table region list -path /maprtable
numberofrows  fid              secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
1237798       2192.32.131254   yarn-94         yarn-92      0                       -INFINITY      56393728     0              \x00\x12\xE3'  56762368
1232985       2194.32.131182   yarn-94         yarn-92      0                       \x00\x12\xE3'  56221696     0              \x00%\xB3\x80  56590336
1247075       2114.147.262606  yarn-92         yarn-94      0                       \x00%\xB3\x80  57008128     0              \x008\xBA\xE3  57376768
6282142       2193.32.131216   yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
12. Merge Regions
Note: You just need to specify the fid of the first region and it will try to merge the it with the next region.
#  maprcli table region list -path /maprtable
numberofrows  fid              secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
1237798       2192.32.131254   yarn-94         yarn-92      0                       -INFINITY      56393728     0              \x00\x12\xE3'  56762368
1232985       2194.32.131182   yarn-94         yarn-92      0                       \x00\x12\xE3'  56221696     0              \x00%\xB3\x80  56590336
1247075       2114.147.262606  yarn-92         yarn-94      0                       \x00%\xB3\x80  57008128     0              \x008\xBA\xE3  57376768
6282142       2193.32.131216   yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
# maprcli table region merge -fid 2194.32.131182 -path /maprtable
#  maprcli table region list -path /maprtable
numberofrows  fid             secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
1237798       2192.32.131254  yarn-94         yarn-92      0                       -INFINITY      56393728     0              \x00\x12\xE3'  56762368
2480060       2194.32.131182  yarn-94         yarn-92      0                       \x00\x12\xE3'  113098752    0              \x008\xBA\xE3  113836032
6282142       2193.32.131216  yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
# maprcli table region merge -fid 2192.32.131254 -path /maprtable
#  maprcli table region list -path /maprtable
numberofrows  fid             secondarynodes  primarynode  numberofrowswithdelete  startkey       logicalsize  lastheartbeat  endkey         physicalsize
3717858       2192.32.131254  yarn-94         yarn-92      0                       -INFINITY      169492480    0              \x008\xBA\xE3  170598400
6282142       2193.32.131216  yarn-94         yarn-92      0                       \x008\xBA\xE3  286400512    0              INFINITY       288317440
If you want to merge the last 2 regions, it will fail if you specify the fid for the last region.
You can only specify the fid of the first region.
# maprcli table region merge -fid 2193.32.131216 -path /maprtable
2015-03-13 06:03:58,0354 ERROR Client fs/client/fileclient/cc/dbclient.cc:585 Thread: 1398 MergeTableRegion failed with error Invalid argument(22) for region 2193.32.131216
ERROR (10003) -  Failed to merge region of table: /maprtable, Error: Invalid argument (22)

# maprcli table region merge -fid 2192.32.131254  -path /maprtable
#  maprcli table region list -path /maprtable
numberofrows  fid             secondarynodes  primarynode  numberofrowswithdelete  startkey   logicalsize  lastheartbeat  endkey    physicalsize
10000000      2192.32.131254  yarn-94         yarn-92      0                       -INFINITY  455892992    0              INFINITY  458915840
13. Disable/Enable Auto Split
maprcli table edit -autosplit false -path /maprtable
maprcli table edit -autosplit true -path /maprtable
14. Set the average region size
maprcli table edit -regionsizemb 256 -path /maprtable
15. YCSB commands
export HBASE_CLASSPATH=$HBASE_CLASSPATH:/root/YCSB/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/root/YCSB/ycsb-0.1.4/core/lib/core-0.1.4.jar
hbase com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -threads 40 -P workloads/workloada -p columnfamily=family -s  -load
hbase com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -threads 40 -P workloads/workloada -p columnfamily=family -s  -t

==

1 comment:

  1. How to find/calculate rowsize in maprdb table for an existing table loaded with data?

    ReplyDelete

Popular Posts