Open-source BigTable implementation on top of HDFS or other distributed FS in the Hadoop ecosystem. Column-based NOSQL database. A stored value is addressed by rowkey, column family, column qualifier, timestamp. From the point of view of the DB rowkey, family, qualifier and value are untyped byte arrays.

An query to the HBase database returning multiple rows. Can be limited by a range of rowkeys, by columns and by timestamp. Can use filters for more control.

OpenTSDB - an interesting monitoring project to (possibly) take some schema ideas from.

HBase Shell

The easiest tool to test HBase, query a table or debug a custom filter or coprocessor.

Java client

The fastest way to query HBase. See test results.

REST interface

HBase provides a REST server that can be run on any node in the cluster (or outside the cluster if desired). It acts as an HBase client and a REST server to provide access to HBase data for non-java clients.

See appendix for more. Advantages:

  • Cross-platform and universal
Main problems:
  • Is supposedly slower than Thrift, not tested
  • Filters and scans can be used with REST, but not custom filters and not Endpoints.

Thrift interface

HBase also provides a Thrift server that can be run on any node in the cluster (or outside the cluster if desired). It acts as an HBase client and a Thrift server to provide access to HBase data for non-java clients.

Thrift is an interface definition language and automatically generates language-specific bindings for the interface.


  • Cross-platform
  • Reported as faster than REST
  • Custom filters can be used if configured on the Thrift server
  • Is slower than a Java client for large scans
  • Language-specific bindings need to be built
  • Endpoints (stored procedure Coprocessors) can't be run with Thrift

HBase from Pig


STORE data INTO 'ssbtest' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:etime, cf:value, cf:color, cf:url, cf:status');

First value in the tuple used as the key. To make a compound key either use several calls to CONCAT or write a UDF

HBase from MapReduce jobs


Scan time test

data 1 month aggregated transfer data, 1.7M rows
key format bTtttVvvvVvvvSsssSsssSsssSsssDdddDdddDdddDddd
salt = hash mod 16, 10 min time bin, vo, src, dst
scan parameters caching = 1000 batch = 100

java client on cluster (dashb-ai-410):

time range vo src dst row count prep scanSorted ascending total
1372637400 1372810200 "atlas1" "" "" 0 5 ms 103 ms 108 ms
1072637400 1672810200 "atlas" "GRIF" "" 417 9 ms 1195 ms 1204 ms
1372637400 1372810200 "atlas" "GRIF" "" 11 5 ms 134 ms 139 ms
1372637400 1372810200 "atlas" "GRIF" "" 11 5 ms 134 ms 139 ms
1372637400 1372810200 "atlas" "GRIF" "GRIF" 2 10 ms 142 ms 152 ms
1072637400 1672810200 "" "GRIF" "" 1400 10 ms 1611 ms 1621 ms
1372637400 1372810200 "atlas" "" "GRIF" 1418 5 ms 270 ms 275 ms
1072637400 1672810200 "" "" "GRIF" 26456 8 ms 4743 ms 4751 ms
1372637400 1372810200 "atlas" "" "" 68970 5 ms 624 ms 629 ms
1372637400 1372810200 "" "" "" 105812 6 ms 831 ms 837 ms
1072637400 1672810200 "" "" "" 1770598     22659 ms

java client on desktop - not tested (exception when creating scan)

thrift client on cluster (dashb-ai-410)

time range vo src dst row count walltime
1372637400 1372810200 "atlas" "" "" 68970 11.0466649532


Phoenix is a tool to provide an interface very close to SQL to query HBase tables. Not for batch processing, not based on MapReduce.


Query language reference

Things we can learn from Phoenix:

  • Key salt
  • Using filters
  • Using coprocessors

Reasons for us not to use Phoenix:

  • JDBC interface, which means Java client. Our existing code is Python.
    • Ways to use JDBC in Python are sketchy, but may warrant another look.
  • Needs to be installed on the cluster.
    • But if we use custom filters and coprocessors ourselves, they would need to be installed too.
Phoenix provide a Pig storage class to store to a Phoenix table from Pig. This should work but fails with an error:
STORE data INTO 'hbase://ssbtest' USING com.salesforce.phoenix.pig.PhoenixHBaseStorage('dashb-ai-410', '-batchSize 10');

Source of the class in question.

One can always use HBase storage for Pig instead. Phoenix tables are just HBase tables with some parameters and coprocessors set. The main disadvantage is concatenating the rowkey value needs to be done manually with HBase storage.


Fliters limit the amount of data retrieved by an HBase scan based on some filtering criteria. Some useful examples: PageFilter (N) is great for debugging, limits the amount of rows returned from a scan to N.

Custom filters

Any filter can be used from HBase shell, provided it's in the classpath. hbase> scan 't1', {FILTER => org.apache.hadoop.hbase.filter.ColumnPaginationFilter.new(1, 0)}

To be able to use filter through Thrift or REST it needs to provide

createFilterFromArguments(ArrayList<byte[]> filterArguments) 

This provides a way to create a filter from string parameters, which is exactly what Thrift/Rest provide. To debug this method for your custom filter from HBase shell, register the method (so that hbase can look up method class name from a short method name) and try a scan like this:

hbase> org.apache.hadoop.hbase.filter.ParseFilter.registerFilter("FuzzyRowFilterFromText", "org.apache.hadoop.hbase.filter.FuzzyRowFilterFromText")
scan 'SSB3', {FILTER => "(FuzzyRowFilterFromText ( '\x00\x00\x00\x00\x00\x00ssb_metric_18\x00\x00\x00VICTORIA-LCG2\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', '\x01\x01\x01\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01') AND (PageFilter (10)) )" }

To use the custom filter with other interfaces, it needs to be registered too:

Adding custom filters to Thrift server configuration.

In CDH4 Cloudera manager:

Services->Hbase1->Configuration->View and edit->Default Category->HBaseThrift Server (Default)->Advanced->HBase Thrift Server Configuration Safety Valve for hbase-site.xml:


This is not implemented for REST yet.

Filters with skip hints

Easy to imagine, harder to understand and implement. Faster than simple filters that just check one row. http://hadoop-hbase.blogspot.ch/2012/01/hbase-intra-row-scanning.html http://hadoop-hbase.blogspot.ch/2012/01/filters-in-hbase-or-intra-row-scanning.html


FuzzyRowFilter is a rowkey filter allowing filter by subsets of the rowkey bytes. Advantages:

  • Fast: provides skip hints to the scanner
  • Structurally simple: just provide keyvalues and byte masks
  • Part of HBase
  • Not very friendly: construct keyvalues and byte masks yourself
  • Does not support ranges for rowkey parts, only discrete values.
    • It's OK in our use-case, we only have one part of the key that needs range filtering: timestamp. We want it to be at the front of the key anyway, so we can use scan's startRow and stopRow to filter by timestamp. Then this filter would only be used to filter by resource and by metric, which don't need range filtering.
  • Does not allow for variable-length rowkey parts.
    • Either we use fixed length of resource and metric strings
    • Or we use binary resource id and metric id to save space
  • Does not implement createFilterFromArguments
    • This is pretty easy to fix with a subclass that implements this method
  • Consequently can't be used with REST or Thrift out of the box (they need createFilterFromArguments)


SkipScanFilter is a rowkey filter allowing to filter by key columns in Phoenix tables. Used iternally by Phoenix queries. Advantages:

  • Fast: provides skip hints to the scanner
  • Supports ranges for key column values.
  • Supports variable-length and empty key columns.
  • Easier to use: for each key column provide a set of value ranges
    • But key columns are defined in Phoenix
  • Part of Phoenix
  • Needs Phoenix metadata (key column parameters) to construct the filter
  • Does not implement createFilterFromArguments
    • Which is harder to implement ourselves because of depending on Phoenix metadata


https://blogs.apache.org/hbase/entry/coprocessor_introduction Observers and endpoints ~ triggers and stored procedures.

Endpoints can't be called through Thrift out of the box: https://issues.apache.org/jira/browse/HBASE-5600

Observers are used by Phoenix to do aggregation, but are not really designed for that. Phoenix coprocessors are too tightly coupled with Phoenix metadata and utilities to be usable from other clients.



UDF (user-defined functions)

UDF are used to do any data processing within PIG more complex that restructuring bags and straightforward aggregation.

Python UDFs. Actually Jython, take care. Not convenient to construct rowkeys because of Unicode issues.

In python requests is a good library for working with REST. Happybase is supposed to be a more high-level HBase-specific library, I have not had time to try it.

Rest interface reference

Filters and scans can be used with REST, but not custom filters and not Endpoints.

Filter parameters reference


<Scanner batch="5"> <filter>{
  "type": "FamilyFilter",
  "op": "EQUAL",
  "comparator": {
    "type": "BinaryComparator",
    "value": "dGVzdHJvdw\u003d\u003d"
}</filter> </Scanner>

Yes, json-like inside xml. Rest also supports application/protobuf instead of XML, but filter definition probably still uses the same format. Note that values are b64 encoded.


Cloudera CDH4


Used by the manager stack

Used by actual servers

Thrift moved from the default 9090 to 7182 (manager port) because we have a lot of ports open as it is. Thrift is running on host 414, manager is on host 410 so there is no conflict as of now.

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r9 - 2013-08-15 - IvanKadochnikov
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback