Goal:
How to enable debug logging for partition pruning phase in Drill.This can help troubleshoot when the partition pruning is not happening due to some reason.
Env:
Drill 1.1Solution:
1. Add below lines in logback.xml on all drill nodes.
<logger name="org.apache.drill.exec.planner.logical.partition.PruneScanRule" additivity="false"> <level value="debug" /> <appender-ref ref="FILE" /> </logger>
2. Restart all drillbits.
3. Run a query which should utilize partition pruning.
For example:> select * from dfs.`/drill/part1` where dir0='2015' and dir1='01'; +----------------+-------+-------+ | columns | dir0 | dir1 | +----------------+-------+-------+ | ["1","'111'"] | 2015 | 01 | +----------------+-------+-------+ 1 row selected (0.219 seconds)Check the foreman drillbit.log, and below debug information should show:
DEBUG o.a.d.e.p.l.partition.PruneScanRule - Attempting to prune AND(=($1, '2015'), =($2, '01')) DEBUG o.a.d.e.p.l.partition.PruneScanRule - Pruned [maprfs:/drill/part1/2015/01/01.csv, maprfs:/drill/part1/2015/02/02.csv, maprfs:/drill/part1/2015/03/03.csv, maprfs:/drill/part1/2014/11/11.csv, maprfs:/drill/part1/2014/12/12.csv] => [maprfs:/drill/part1/2015/01/01.csv]
No comments:
Post a Comment