Skip to content

Commit

Permalink
Version updated to v1.4. Expanded README with limitations and examples.
Browse files Browse the repository at this point in the history
  • Loading branch information
m-manu committed Sep 15, 2016
1 parent 56b33b6 commit fe3a21d
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 17 deletions.
52 changes: 36 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@
## Introduction
This compact utility library is an annotation based *object mapper* for HBase (written in Java) that helps you:

* convert objects of your bean-like classes to HBase rows and vice-versa ➞ for use in MapReduce jobs on HBase tables and their unit-tests
* define *data access objects* for entities that map to HBase rows ➞ for random single/range/bulk access of rows of an HBase table
* convert objects of your bean-like classes to HBase rows and vice-versa
* for use in Hadoop MapReduce jobs that read from and/or write to HBase tables
* and write efficient unit-tests for `Mapper` and `Reducer` classes
* define *data access objects* for entities that map to HBase rows
* for random single/range/bulk access of rows of an HBase table

## Usage
Let's say you've an HBase table `citizens` with row-key format `country_code#UID`. Let's say your table is created with two column families `main` and `optional` which may have columns like `uid`, `name`, `salary` etc.
Let's say you've an HBase table `citizens` with row-key format of `country_code#UID`. Now, let's say your table is created with two column families `main` and `optional`, which may have columns like `uid`, `name`, `salary` etc.

This library enables to you represent your HBase table as a class, like below:
This library enables to you represent your HBase table as a bean-like class, as below:

```java
@HBTable("citizens")
public class Citizen implements HBRecord<String> {
// Generic type argument above indicates the data type to which you want to map your HBase row key
// In this case, I'm mapping it as a String

@HBRowKey
private String countryCode;
@HBRowKey
Expand All @@ -43,9 +43,20 @@ public class Citizen implements HBRecord<String> {
this.countryCode = pieces[0];
this.uid = Integer.parseInt(pieces[1]);
}

// Constructors, getters and setters
}
```
(see [Citizen.java](./src/test/java/com/flipkart/hbaseobjectmapper/entities/Citizen.java) for a detailed example with lot many data types)
where, following things are provided:

* Name of the HBase table (`citizens`) that the class maps to, using `HBTable` annotation
* Data type to map row keys to (`String`) as generic type parameter for `HBRecord` interface
* Logics for conversion of HBase row key to Java types and vice-versa by implmenting `parseRowKey` and `composeRowKey` methods
* Names of columns and their column families using `HBColumn` annotation

This library enables you to represent rows of `citizens` HBase table as instances of `Citizen` class.

See source files [Citizen.java](./src/test/java/com/flipkart/hbaseobjectmapper/entities/Citizen.java) and [Employee.java](./src/test/java/com/flipkart/hbaseobjectmapper/entities/Employee.java) for detailed examples.

Now, for above definition of your `Citizen` class,

Expand All @@ -55,7 +66,7 @@ Now, for above definition of your `Citizen` class,
## MapReduce use-cases

### Mapper
If your MapReduce job is reading from an HBase table, in your `map()` method, HBase's `Result` object can be converted to your object of your bean-like class using below method:
If your MapReduce job is reading from an HBase table, in your `map()` method, HBase's `Result` object can be converted to object of your bean-like class using below method:

```java
<T extends HBRecord> T readValue(ImmutableBytesWritable rowKey, Result result, Class<T> clazz)
Expand Down Expand Up @@ -89,7 +100,7 @@ See file [CitizenReducer.java](./src/test/java/com/flipkart/hbaseobjectmapper/mr
### Unit-test for Mapper
If your MapReduce job is reading from an HBase table, you would want to unit-test your `map()` method as below.

Your bean-like object can be converted to HBase's `Put` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:
Object of your bean-like class can be converted to HBase's `Put` (for row contents) and `ImmutableBytesWritable` (for row key) using below methods:

```java
ImmutableBytesWritable getRowKey(HBRecord obj)
Expand Down Expand Up @@ -129,7 +140,7 @@ Pair<ImmutableBytesWritable, Writable> reducerResult = reducerDriver.withInput(U
Citizen citizen = hbObjectMapper.readValue(reducerResult.getFirst(), (Put) reducerResult.getSecond(), Citizen.class);
```

See file [TestCitizenReducer.java](./src/test/java/com/flipkart/hbaseobjectmapper/mr/TestCitizenReducer.java) for full sample code that unit-tests a reducer using [MRUnit](https://mrunit.apache.org/)
See file [TestCitizenReducer.java](./src/test/java/com/flipkart/hbaseobjectmapper/mr/TestCitizenReducer.java) for full sample code.

## HBase Random Access
This library provides an abstract class to define your own *data access object*. For example you can create a *data access object* for `Citizen` class in the above example as follows:
Expand Down Expand Up @@ -179,9 +190,18 @@ citizenDao.delete("IND#2"); // Delete a row by it's row key
citizenDao.getHBaseTable() // returns HTable instance (in case you want to directly play around)

```
(see [TestsAbstractHBDAO.java](./src/test/java/com/flipkart/hbaseobjectmapper/TestsAbstractHBDAO.java) for a more detailed example)
(see [TestsAbstractHBDAO.java](./src/test/java/com/flipkart/hbaseobjectmapper/TestsAbstractHBDAO.java) for a more detailed examples)

**Please note:** Since we're dealing with HBase (and not an OLTP data store), fitting an classical ORM paradigm may not make sense. So this library doesn't intend to evolve as a full-fledged ORM. However, if you do intend to use HBase via ORM, I suggest you use [Apache Phoenix](https://phoenix.apache.org/).


## Limitations

**Please note:** Since we're dealing with HBase (and not an OLTP data store), fitting an ORM paradigm may not make sense. But, if you do intend to use HBase via ORM, I suggest you use [Apache Phoenix](https://phoenix.apache.org/). This library, however, doesn't intend to evolve as a full-fledged ORM.
* Being an *object mapper*, this library works for pre-defined columns only. For example, this library doesn't provide ways to fetch:
* columns matching a regular expression
* (unmapped) columns of a column family
* When you `get` a row by it's row-key, data for all columns (of all families) are fetched.
* Workaround: If you want to selectively fetch data for given column families, use multiple classes, one for each column family


## Maven
Expand All @@ -191,16 +211,16 @@ Add below entry within the `dependencies` section of your `pom.xml`:
<dependency>
<groupId>com.flipkart</groupId>
<artifactId>hbase-object-mapper</artifactId>
<version>1.3</version>
<version>1.4</version>
</dependency>
```
(See artifact details for [com.flipkart:hbase-object-mapper:1.3]((http://search.maven.org/#artifactdetails%7Ccom.flipkart%7Chbase-object-mapper%7C1.3%7Cjar)) on **Maven Central**)
(See artifact details for [com.flipkart:hbase-object-mapper:1.4]((http://search.maven.org/#artifactdetails%7Ccom.flipkart%7Chbase-object-mapper%7C1.4%7Cjar)) on **Maven Central**)

## How to build?
To build this project, follow below steps:

* Do a `git clone` of this repository
* Checkout latest stable version `git checkout v1.3`
* Checkout latest stable version `git checkout v1.4`
* Execute `mvn clean install` from shell

Currently, projects that use this library are running on [Hortonworks Data Platform v2.2](http://hortonworks.com/blog/announcing-hdp-2-2/) (corresponds to Hadoop 2.6 and HBase 0.98). However, if you're using a different distribution of Hadoop (like [Cloudera](http://www.cloudera.com/)) or if you are using a different version of Hadoop, you may change the versions in [pom.xml](./pom.xml) to desired ones and build the project.
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.flipkart</groupId>
<artifactId>hbase-object-mapper</artifactId>
<version>1.4-SNAPSHOT</version>
<version>1.4</version>
<url>https://github.com/flipkart-incubator/hbase-object-mapper#readme</url>
<scm>
<url>https://github.com/flipkart-incubator/hbase-object-mapper</url>
Expand Down

0 comments on commit fe3a21d

Please sign in to comment.