Observer协处理器通常在一个特定的事件(诸如Get
或Put
)之前或之后发生,相当于RDBMS中的触发器。Endpoint协处理器则类似于RDBMS中的存储过程,因为它可以让你在RegionServer上对数据执行自定义计算,而不是在客户端上执行计算。
使用与Thrift类似——
本文是以上两者的简单实例,使用的环境:环境 jdk1.8 Hadoop2.6.5 hbase1.2.4。
1、Endpoint实例
1>
编写适用于protobuf的proto文件,如下,尽量不要带注释,因为编译时可能出现乱码
下载EXE
option java_package = "com.endpoint.test";
option java_outer_classname = "Sum";
option java_generic_services = true;
option java_generate_equals_and_hash = true;
option optimize_for = SPEED;
message SumRequest {
required string family = 1;
required string column = 2;
}
message SumResponse {
required int64 sum = 1 [default = 0];
}
service SumService {
rpc getSum(SumRequest)
returns (SumResponse);
}
https://developers.google.com/protocol-buffers/docs/downloads
2> 编译上面的proto文件
使用protoc程序进行编译,linux下或者windows均可,protoc程序可以直接从github下载:https://github.com/google/protobuf/releases,也可以自己编译生成,参见
CentOS
7下protobuf的源码编译安装 见
http://www.linuxidc.com/Linux/2016-12/138716.htm。
注意,编译的版本要与hadoop以及hbase使用的版本相同,或者略高,但最好不要过高,hadoop2.6.5 hbase1.2.4使用的都是protobuf2.5.0的版本,写此篇文章时的最新版为3.1.0
Demo地址
(高版本必须指定syntax,例如proto3的syntax在第一行非空白非注释行,必须写:syntax = "proto3",字段规则移除了 “required”,并把 “optional” 改名为 “singular”,移除了 default 选项。可搜索Protobuf 的 proto3 与 proto2 的区别进行了解。)下载的话选择带win或linux的版本,这是编译好的版本。有很多带具体语言的版本,是一些具体某种语言的发行版源码包。,为了与hbase以及hadoop统一起来,此处用的是protoc-2.5.0-win32.zip。
https://developers.google.com/protocol-buffers/docs/javatutorial#compiling-your-protocol-buffers
解压文件:
中有addressbook.proto内容
使用windows命令行进入上面的目录,执行以下命令即可:
举例子:放在
protoc.exe sum1.proto --java_out=./
D:UsersggDesktopprotoc-2.6.1-win32
高版本有编译好的适用于linux下的protoc程序文件,低版本没有。在linux下执行以下命令:
protoc sum.proto --java_out=./
address.proto如下:
结果都一样,生成的代码参见折叠部分,有很多,因为上面文件中指定java_outer_classname = "Sum",所以会生成Sum类,将这个类引入到项目中,注意项目的包名称与上面文件中指定(option java_package = "com.endpoint.test")的名称要一致。
package tutorial;
// Generated by the protocol buffer compiler. DO NOT EDIT!
// source: sumcode.proto
package com.endpoint.test;
option java_package = "com.example.tutorial";
public final class Sum {
private Sum() {}
public static void registerAllExtensions(
com.google.protobuf.ExtensionRegistry registry) {
}
public interface SumRequestOrBuilder
extends com.google.protobuf.MessageOrBuilder {
option java_outer_classname = "AddressBookProtos";
// required string family = 1;
/**
* <code>required string family = 1;</code>
*/
boolean hasFamily();
/**
* <code>required string family = 1;</code>
*/
java.lang.String getFamily();
/**
* <code>required string family = 1;</code>
*/
com.google.protobuf.ByteString
getFamilyBytes();
// required string column = 2;
/**
* <code>required string column = 2;</code>
*/
boolean hasColumn();
/**
* <code>required string column = 2;</code>
*/
java.lang.String getColumn();
/**
* <code>required string column = 2;</code>
*/
com.google.protobuf.ByteString
getColumnBytes();
}
/**
* Protobuf type {@code SumRequest}
*/
public static final class SumRequest extends
com.google.protobuf.GeneratedMessage
implements SumRequestOrBuilder {
// Use SumRequest.newBuilder() to construct.
private
SumRequest(com.google.protobuf.GeneratedMessage.Builder<?>
builder) {
super(builder);
this.unknownFields = builder.getUnknownFields();
}
private SumRequest(boolean noInit) { this.unknownFields =
com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
message Person {
private static final SumRequest defaultInstance;
public static SumRequest getDefaultInstance() {
return defaultInstance;
}
required string name = 1;
public SumRequest getDefaultInstanceForType() {
return defaultInstance;
}
required int32 id = 2;
private final com.google.protobuf.UnknownFieldSet unknownFields;
@java.lang.Override
public final com.google.protobuf.UnknownFieldSet
getUnknownFields() {
return this.unknownFields;
}
private SumRequest(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
initFields();
int mutable_bitField0_ = 0;
com.google.protobuf.UnknownFieldSet.Builder unknownFields =
com.google.protobuf.UnknownFieldSet.newBuilder();
try {
boolean done = false;
while (!done) {
int tag = input.readTag();
switch (tag) {
case 0:
done = true;
break;
default: {
if (!parseUnknownField(input, unknownFields,
extensionRegistry, tag)) {
done = true;
}
break;
}
case 10: {
bitField0_ |= 0x00000001;
family_ = input.readBytes();
break;
}
case 18: {
bitField0_ |= 0x00000002;
column_ = input.readBytes();
break;
}
}
}
} catch (com.google.protobuf.InvalidProtocolBufferException e) {
throw e.setUnfinishedMessage(this);
} catch (java.io.IOException e) {
throw new com.google.protobuf.InvalidProtocolBufferException(
e.getMessage()).setUnfinishedMessage(this);
} finally {
this.unknownFields = unknownFields.build();
makeExtensionsImmutable();
}
}
public static final com.google.protobuf.Descriptors.Descriptor
getDescriptor() {
return
com.endpoint.test.Sum.internal_static_SumRequest_descriptor;
}
optional string email = 3;
protected com.google.protobuf.GeneratedMessage.FieldAccessorTable
internalGetFieldAccessorTable() {
return
com.endpoint.test.Sum.internal_static_SumRequest_fieldAccessorTable
.ensureFieldAccessorsInitialized(
com.endpoint.test.Sum.SumRequest.class,
com.endpoint.test.Sum.SumRequest.Builder.class);
}
public static com.google.protobuf.Parser<SumRequest> PARSER
new com.google.protobuf.AbstractParser<SumRequest>() {
public SumRequest parsePartialFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return new SumRequest(input, extensionRegistry);
}
};
@java.lang.Override
public com.google.protobuf.Parser<SumRequest>
getParserForType() {
return PARSER;
}
private int bitField0_;
// required string family = 1;
public static final int FAMILY_FIELD_NUMBER = 1;
private java.lang.Object family_;
/**
* <code>required string family = 1;</code>
*/
public boolean hasFamily() {
return ((bitField0_ & 0x00000001) == 0x00000001);
}
/**
* <code>required string family = 1;</code>
*/
public java.lang.String getFamily() {
java.lang.Object ref = family_;
if (ref instanceof java.lang.String) {
return (java.lang.String) ref;
} else {
com.google.protobuf.ByteString bs =
(com.google.protobuf.ByteString) ref;
java.lang.String s = bs.toStringUtf8();
if (bs.isValidUtf8()) {
family_ = s;
}
return s;
}
}
/**
* <code>required string family = 1;</code>
*/
public com.google.protobuf.ByteString
getFamilyBytes() {
java.lang.Object ref = family_;
if (ref instanceof java.lang.String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
family_ = b;
return b;
} else {
return (com.google.protobuf.ByteString) ref;
}
}
// required string column = 2;
public static final int COLUMN_FIELD_NUMBER = 2;
private java.lang.Object column_;
/**
* <code>required string column = 2;</code>
*/
public boolean hasColumn() {
return ((bitField0_ & 0x00000002) == 0x00000002);
}
/**
* <code>required string column = 2;</code>
*/
public java.lang.String getColumn() {
java.lang.Object ref = column_;
if (ref instanceof java.lang.String) {
return (java.lang.String) ref;
} else {
com.google.protobuf.ByteString bs =
(com.google.protobuf.ByteString) ref;
java.lang.String s = bs.toStringUtf8();
if (bs.isValidUtf8()) {
column_ = s;
}
return s;
}
}
/**
* <code>required string column = 2;</code>
*/
public com.google.protobuf.ByteString
getColumnBytes() {
java.lang.Object ref = column_;
if (ref instanceof java.lang.String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
column_ = b;
return b;
} else {
return (com.google.protobuf.ByteString) ref;
}
}
private void initFields() {
family_ = "";
column_ = "";
}
private byte memoizedIsInitialized = -1;
public final boolean isInitialized() {
byte isInitialized = memoizedIsInitialized;
if (isInitialized != -1) return isInitialized == 1;
if (!hasFamily()) {
memoizedIsInitialized = 0;
return false;
}
if (!hasColumn()) {
memoizedIsInitialized = 0;
return false;
}
memoizedIsInitialized = 1;
return true;
}
public void writeTo(com.google.protobuf.CodedOutputStream output)
throws java.io.IOException {
getSerializedSize();
if (((bitField0_ & 0x00000001) == 0x00000001)) {
output.writeBytes(1, getFamilyBytes());
}
if (((bitField0_ & 0x00000002) == 0x00000002)) {
output.writeBytes(2, getColumnBytes());
}
getUnknownFields().writeTo(output);
}
private int memoizedSerializedSize = -1;
public int getSerializedSize() {
int size = memoizedSerializedSize;
if (size != -1) return size;
size = 0;
if (((bitField0_ & 0x00000001) == 0x00000001)) {
size = com.google.protobuf.CodedOutputStream
.computeBytesSize(1, getFamilyBytes());
}
if (((bitField0_ & 0x00000002) == 0x00000002)) {
size = com.google.protobuf.CodedOutputStream
.computeBytesSize(2, getColumnBytes());
}
size = getUnknownFields().getSerializedSize();
memoizedSerializedSize = size;
return size;
}
private static final long serialVersionUID = 0L;
@java.lang.Override
protected java.lang.Object writeReplace()
throws java.io.ObjectStreamException {
return super.writeReplace();
}
@java.lang.Override
public boolean equals(final java.lang.Object obj) {
if (obj == this) {
return true;
}
if (!(obj instanceof com.endpoint.test.Sum.SumRequest)) {
return super.equals(obj);
}
com.endpoint.test.Sum.SumRequest other =
(com.endpoint.test.Sum.SumRequest) obj;
boolean result = true;
result = result && (hasFamily() == other.hasFamily());
if (hasFamily()) {
result = result && getFamily()
.equals(other.getFamily());
}
result = result && (hasColumn() == other.hasColumn());
if (hasColumn()) {
result = result && getColumn()
.equals(other.getColumn());
}
result = result &&
getUnknownFields().equals(other.getUnknownFields());
return result;
}
private int memoizedHashCode = 0;
@java.lang.Override
public int hashCode() {
if (memoizedHashCode != 0) {
return memoizedHashCode;
}
int hash = 41;
hash = (19 * hash) getDescriptorForType().hashCode();
if (hasFamily()) {
hash = (37 * hash) FAMILY_FIELD_NUMBER;
hash = (53 * hash) getFamily().hashCode();
}
if (hasColumn()) {
hash = (37 * hash) COLUMN_FIELD_NUMBER;
hash = (53 * hash) getColumn().hashCode();
}
hash = (29 * hash) getUnknownFields().hashCode();
memoizedHashCode = hash;
return hash;
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
com.google.protobuf.ByteString data)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
com.google.protobuf.ByteString data,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data, extensionRegistry);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(byte[]
data)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
byte[] data,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data, extensionRegistry);
}
public static com.endpoint.test.Sum.SumRequest
parseFrom(java.io.InputStream input)
throws java.io.IOException {
return PARSER.parseFrom(input);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
java.io.InputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseFrom(input, extensionRegistry);
}
public static com.endpoint.test.Sum.SumRequest
parseDelimitedFrom(java.io.InputStream input)
throws java.io.IOException {
return PARSER.parseDelimitedFrom(input);
}
public static com.endpoint.test.Sum.SumRequest parseDelimitedFrom(
java.io.InputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseDelimitedFrom(input, extensionRegistry);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
com.google.protobuf.CodedInputStream input)
throws java.io.IOException {
return PARSER.parseFrom(input);
}
public static com.endpoint.test.Sum.SumRequest parseFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseFrom(input, extensionRegistry);
}
public static Builder newBuilder() { return Builder.create(); }
public Builder newBuilderForType() { return newBuilder(); }
public static Builder newBuilder(com.endpoint.test.Sum.SumRequest
prototype) {
return newBuilder().mergeFrom(prototype);
}
public Builder toBuilder() { return newBuilder(this); }
@java.lang.Override
protected Builder newBuilderForType(
com.google.protobuf.GeneratedMessage.BuilderParent parent) {
Builder builder = new Builder(parent);
return builder;
}
/**
* Protobuf type {@code SumRequest}
*/
public static final class Builder extends
com.google.protobuf.GeneratedMessage.Builder<Builder>
implements com.endpoint.test.Sum.SumRequestOrBuilder {
public static final com.google.protobuf.Descriptors.Descriptor
getDescriptor() {
return
com.endpoint.test.Sum.internal_static_SumRequest_descriptor;
}
protected
com.google.protobuf.GeneratedMessage.FieldAccessorTable
internalGetFieldAccessorTable() {
return
com.endpoint.test.Sum.internal_static_SumRequest_fieldAccessorTable
.ensureFieldAccessorsInitialized(
com.endpoint.test.Sum.SumRequest.class,
com.endpoint.test.Sum.SumRequest.Builder.class);
}
// Construct using com.endpoint.test.Sum.SumRequest.newBuilder()
private Builder() {
maybeForceBuilderInitialization();
}
private Builder(
com.google.protobuf.GeneratedMessage.BuilderParent parent) {
super(parent);
maybeForceBuilderInitialization();
}
private void maybeForceBuilderInitialization() {
if
(com.google.protobuf.GeneratedMessage.alwaySUSEFieldBuilders)
{
}
}
private static Builder create() {
return new Builder();
}
public Builder clear() {
super.clear();
family_ = "";
bitField0_ = (bitField0_ & ~0x00000001);
column_ = "";
bitField0_ = (bitField0_ & ~0x00000002);
return this;
}
public Builder clone() {
return create().mergeFrom(buildPartial());
}
public com.google.protobuf.Descriptors.Descriptor
getDescriptorForType() {
return
com.endpoint.test.Sum.internal_static_SumRequest_descriptor;
}
public com.endpoint.test.Sum.SumRequest
getDefaultInstanceForType() {
return com.endpoint.test.Sum.SumRequest.getDefaultInstance();
}
public com.endpoint.test.Sum.SumRequest build() {
com.endpoint.test.Sum.SumRequest result = buildPartial();
if (!result.isInitialized()) {
throw newUninitializedMessageException(result);
}
return result;
}
public com.endpoint.test.Sum.SumRequest buildPartial() {
com.endpoint.test.Sum.SumRequest result = new
com.endpoint.test.Sum.SumRequest(this);
int from_bitField0_ = bitField0_;
int to_bitField0_ = 0;
if (((from_bitField0_ & 0x00000001) == 0x00000001)) {
to_bitField0_ |= 0x00000001;
}
result.family_ = family_;
if (((from_bitField0_ & 0x00000002) == 0x00000002)) {
to_bitField0_ |= 0x00000002;
}
result.column_ = column_;
result.bitField0_ = to_bitField0_;
onBuilt();
return result;
}
public Builder mergeFrom(com.google.protobuf.Message other) {
if (other instanceof com.endpoint.test.Sum.SumRequest) {
return mergeFrom((com.endpoint.test.Sum.SumRequest)other);
} else {
super.mergeFrom(other);
return this;
}
}
public Builder mergeFrom(com.endpoint.test.Sum.SumRequest other)
{
if (other ==
com.endpoint.test.Sum.SumRequest.getDefaultInstance()) return this;
if (other.hasFamily()) {
bitField0_ |= 0x00000001;
family_ = other.family_;
onChanged();
}
if (other.hasColumn()) {
bitField0_ |= 0x00000002;
column_ = other.column_;
onChanged();
}
this.mergeUnknownFields(other.getUnknownFields());
return this;
}
public final boolean isInitialized() {
if (!hasFamily()) {
return false;
}
if (!hasColumn()) {
return false;
}
return true;
}
public Builder mergeFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
com.endpoint.test.Sum.SumRequest parsedMessage = null;
try {
parsedMessage = PARSER.parsePartialFrom(input,
extensionRegistry);
} catch (com.google.protobuf.InvalidProtocolBufferException e)
{
parsedMessage = (com.endpoint.test.Sum.SumRequest)
e.getUnfinishedMessage();
throw e;
} finally {
if (parsedMessage != null) {
mergeFrom(parsedMessage);
}
}
return this;
}
private int bitField0_;
// required string family = 1;
private java.lang.Object family_ = "";
/**
* <code>required string family = 1;</code>
*/
public boolean hasFamily() {
return ((bitField0_ & 0x00000001) == 0x00000001);
}
/**
* <code>required string family = 1;</code>
*/
public java.lang.String getFamily() {
java.lang.Object ref = family_;
if (!(ref instanceof java.lang.String)) {
java.lang.String s = ((com.google.protobuf.ByteString) ref)
.toStringUtf8();
family_ = s;
return s;
} else {
return (java.lang.String) ref;
}
}
/**
* <code>required string family = 1;</code>
*/
public com.google.protobuf.ByteString
getFamilyBytes() {
java.lang.Object ref = family_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
family_ = b;
return b;
} else {
return (com.google.protobuf.ByteString) ref;
}
}
/**
* <code>required string family = 1;</code>
*/
public Builder setFamily(
java.lang.String value) {
if (value == null) {
throw new NullPointerException();
}
bitField0_ |= 0x00000001;
family_ = value;
onChanged();
return this;
}
/**
* <code>required string family = 1;</code>
*/
public Builder clearFamily() {
bitField0_ = (bitField0_ & ~0x00000001);
family_ = getDefaultInstance().getFamily();
onChanged();
return this;
}
/**
* <code>required string family = 1;</code>
*/
public Builder setFamilyBytes(
com.google.protobuf.ByteString value) {
if (value == null) {
throw new NullPointerException();
}
bitField0_ |= 0x00000001;
family_ = value;
onChanged();
return this;
}
// required string column = 2;
private java.lang.Object column_ = "";
/**
* <code>required string column = 2;</code>
*/
public boolean hasColumn() {
return ((bitField0_ & 0x00000002) == 0x00000002);
}
/**
* <code>required string column = 2;</code>
*/
public java.lang.String getColumn() {
java.lang.Object ref = column_;
if (!(ref instanceof java.lang.String)) {
java.lang.String s = ((com.google.protobuf.ByteString) ref)
.toStringUtf8();
column_ = s;
return s;
} else {
return (java.lang.String) ref;
}
}
/**
* <code>required string column = 2;</code>
*/
public com.google.protobuf.ByteString
getColumnBytes() {
java.lang.Object ref = column_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
column_ = b;
return b;
} else {
return (com.google.protobuf.ByteString) ref;
}
}
/**
*wns9778.com, <code>required string column = 2;</code>
*/
public Builder setColumn(
java.lang.String value) {
if (value == null) {
throw new NullPointerException();
}
bitField0_ |= 0x00000002;
column_ = value;
onChanged();
return this;
}
/**
* <code>required string column = 2;</code>
*/
public Builder clearColumn() {
bitField0_ = (bitField0_ & ~0x00000002);
column_ = getDefaultInstance().getColumn();
onChanged();
return this;
}
/**
* <code>required string column = 2;</code>
*/
public Builder setColumnBytes(
com.google.protobuf.ByteString value) {
if (value == null) {
throw new NullPointerException();
}
bitField0_ |= 0x00000002;
column_ = value;
onChanged();
return this;
}
// @@protoc_insertion_point(builder_scope:SumRequest)
}
static {
defaultInstance = new SumRequest(true);
defaultInstance.initFields();
}
// @@protoc_insertion_point(class_scope:SumRequest)
}
public interface SumResponseOrBuilder
extends com.google.protobuf.MessageOrBuilder {
// required int64 sum = 1 [default = 0];
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
boolean hasSum();
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
long getSum();
}
/**
* Protobuf type {@code SumResponse}
*/
public static final class SumResponse extends
com.google.protobuf.GeneratedMessage
implements SumResponseOrBuilder {
// Use SumResponse.newBuilder() to construct.
private
SumResponse(com.google.protobuf.GeneratedMessage.Builder<?>
builder) {
super(builder);
this.unknownFields = builder.getUnknownFields();
}
private SumResponse(boolean noInit) { this.unknownFields =
com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
private static final SumResponse defaultInstance;
public static SumResponse getDefaultInstance() {
return defaultInstance;
}
public SumResponse getDefaultInstanceForType() {
return defaultInstance;
}
private final com.google.protobuf.UnknownFieldSet unknownFields;
@java.lang.Override
public final com.google.protobuf.UnknownFieldSet
getUnknownFields() {
return this.unknownFields;
}
private SumResponse(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
initFields();
int mutable_bitField0_ = 0;
com.google.protobuf.UnknownFieldSet.Builder unknownFields =
com.google.protobuf.UnknownFieldSet.newBuilder();
try {
boolean done = false;
while (!done) {
int tag = input.readTag();
switch (tag) {
case 0:
done = true;
break;
default: {
if (!parseUnknownField(input, unknownFields,
extensionRegistry, tag)) {
done = true;
}
break;
}
case 8: {
bitField0_ |= 0x00000001;
sum_ = input.readInt64();
break;
}
}
}
} catch (com.google.protobuf.InvalidProtocolBufferException e) {
throw e.setUnfinishedMessage(this);
} catch (java.io.IOException e) {
throw new com.google.protobuf.InvalidProtocolBufferException(
e.getMessage()).setUnfinishedMessage(this);
} finally {
this.unknownFields = unknownFields.build();
makeExtensionsImmutable();
}
}
public static final com.google.protobuf.Descriptors.Descriptor
getDescriptor() {
return
com.endpoint.test.Sum.internal_static_SumResponse_descriptor;
}
protected com.google.protobuf.GeneratedMessage.FieldAccessorTable
internalGetFieldAccessorTable() {
return
com.endpoint.test.Sum.internal_static_SumResponse_fieldAccessorTable
.ensureFieldAccessorsInitialized(
com.endpoint.test.Sum.SumResponse.class,
com.endpoint.test.Sum.SumResponse.Builder.class);
}
enum PhoneType {
public static com.google.protobuf.Parser<SumResponse> PARSER
new com.google.protobuf.AbstractParser<SumResponse>() {
public SumResponse parsePartialFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return new SumResponse(input, extensionRegistry);
}
};
@java.lang.Override
public com.google.protobuf.Parser<SumResponse>
getParserForType() {
return PARSER;
}
private int bitField0_;
// required int64 sum = 1 [default = 0];
public static final int SUM_FIELD_NUMBER = 1;
private long sum_;
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public boolean hasSum() {
return ((bitField0_ & 0x00000001) == 0x00000001);
}
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public long getSum() {
return sum_;
}
private void initFields() {
sum_ = 0L;
}
private byte memoizedIsInitialized = -1;
public final boolean isInitialized() {
byte isInitialized = memoizedIsInitialized;
if (isInitialized != -1) return isInitialized == 1;
if (!hasSum()) {
memoizedIsInitialized = 0;
return false;
}
memoizedIsInitialized = 1;
return true;
}
public void writeTo(com.google.protobuf.CodedOutputStream output)
throws java.io.IOException {
getSerializedSize();
if (((bitField0_ & 0x00000001) == 0x00000001)) {
output.writeInt64(1, sum_);
}
getUnknownFields().writeTo(output);
}
private int memoizedSerializedSize = -1;
public int getSerializedSize() {
int size = memoizedSerializedSize;
if (size != -1) return size;
size = 0;
if (((bitField0_ & 0x00000001) == 0x00000001)) {
size = com.google.protobuf.CodedOutputStream
.computeInt64Size(1, sum_);
}
size = getUnknownFields().getSerializedSize();
memoizedSerializedSize = size;
return size;
}
private static final long serialVersionUID = 0L;
@java.lang.Override
protected java.lang.Object writeReplace()
throws java.io.ObjectStreamException {
return super.writeReplace();
}
@java.lang.Override
public boolean equals(final java.lang.Object obj) {
if (obj == this) {
return true;
}
if (!(obj instanceof com.endpoint.test.Sum.SumResponse)) {
return super.equals(obj);
}
com.endpoint.test.Sum.SumResponse other =
(com.endpoint.test.Sum.SumResponse) obj;
boolean result = true;
result = result && (hasSum() == other.hasSum());
if (hasSum()) {
result = result && (getSum()
== other.getSum());
}
result = result &&
getUnknownFields().equals(other.getUnknownFields());
return result;
}
private int memoizedHashCode = 0;
@java.lang.Override
public int hashCode() {
if (memoizedHashCode != 0) {
return memoizedHashCode;
}
int hash = 41;
hash = (19 * hash) getDescriptorForType().hashCode();
if (hasSum()) {
hash = (37 * hash) SUM_FIELD_NUMBER;
hash = (53 * hash) hashLong(getSum());
}
hash = (29 * hash) getUnknownFields().hashCode();
memoizedHashCode = hash;
return hash;
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
com.google.protobuf.ByteString data)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
com.google.protobuf.ByteString data,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data, extensionRegistry);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(byte[]
data)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
byte[] data,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
return PARSER.parseFrom(data, extensionRegistry);
}
public static com.endpoint.test.Sum.SumResponse
parseFrom(java.io.InputStream input)
throws java.io.IOException {
return PARSER.parseFrom(input);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
java.io.InputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseFrom(input, extensionRegistry);
}
public static com.endpoint.test.Sum.SumResponse
parseDelimitedFrom(java.io.InputStream input)
throws java.io.IOException {
return PARSER.parseDelimitedFrom(input);
}
public static com.endpoint.test.Sum.SumResponse
parseDelimitedFrom(
java.io.InputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseDelimitedFrom(input, extensionRegistry);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
com.google.protobuf.CodedInputStream input)
throws java.io.IOException {
return PARSER.parseFrom(input);
}
public static com.endpoint.test.Sum.SumResponse parseFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
return PARSER.parseFrom(input, extensionRegistry);
}
public static Builder newBuilder() { return Builder.create(); }
public Builder newBuilderForType() { return newBuilder(); }
public static Builder newBuilder(com.endpoint.test.Sum.SumResponse
prototype) {
return newBuilder().mergeFrom(prototype);
}
public Builder toBuilder() { return newBuilder(this); }
@java.lang.Override
protected Builder newBuilderForType(
com.google.protobuf.GeneratedMessage.BuilderParent parent) {
Builder builder = new Builder(parent);
return builder;
}
/**
* Protobuf type {@code SumResponse}
*/
public static final class Builder extends
com.google.protobuf.GeneratedMessage.Builder<Builder>
implements com.endpoint.test.Sum.SumResponseOrBuilder {
public static final com.google.protobuf.Descriptors.Descriptor
getDescriptor() {
return
com.endpoint.test.Sum.internal_static_SumResponse_descriptor;
}
protected
com.google.protobuf.GeneratedMessage.FieldAccessorTable
internalGetFieldAccessorTable() {
return
com.endpoint.test.Sum.internal_static_SumResponse_fieldAccessorTable
.ensureFieldAccessorsInitialized(
com.endpoint.test.Sum.SumResponse.class,
com.endpoint.test.Sum.SumResponse.Builder.class);
}
// Construct using
com.endpoint.test.Sum.SumResponse.newBuilder()
private Builder() {
maybeForceBuilderInitialization();
}
private Builder(
com.google.protobuf.GeneratedMessage.BuilderParent parent) {
super(parent);
maybeForceBuilderInitialization();
}
private void maybeForceBuilderInitialization() {
if (com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders)
{
}
}
private static Builder create() {
return new Builder();
}
public Builder clear() {
super.clear();
sum_ = 0L;
bitField0_ = (bitField0_ & ~0x00000001);
return this;
}
public Builder clone() {
return create().mergeFrom(buildPartial());
}
public com.google.protobuf.Descriptors.Descriptor
getDescriptorForType() {
return
com.endpoint.test.Sum.internal_static_SumResponse_descriptor;
}
public com.endpoint.test.Sum.SumResponse
getDefaultInstanceForType() {
return com.endpoint.test.Sum.SumResponse.getDefaultInstance();
}
public com.endpoint.test.Sum.SumResponse build() {
com.endpoint.test.Sum.SumResponse result = buildPartial();
if (!result.isInitialized()) {
throw newUninitializedMessageException(result);
}
return result;
}
public com.endpoint.test.Sum.SumResponse buildPartial() {
com.endpoint.test.Sum.SumResponse result = new
com.endpoint.test.Sum.SumResponse(this);
int from_bitField0_ = bitField0_;
int to_bitField0_ = 0;
if (((from_bitField0_ & 0x00000001) == 0x00000001)) {
to_bitField0_ |= 0x00000001;
}
result.sum_ = sum_;
result.bitField0_ = to_bitField0_;
onBuilt();
return result;
}
public Builder mergeFrom(com.google.protobuf.Message other) {
if (other instanceof com.endpoint.test.Sum.SumResponse) {
return mergeFrom((com.endpoint.test.Sum.SumResponse)other);
} else {
super.mergeFrom(other);
return this;
}
}
public Builder mergeFrom(com.endpoint.test.Sum.SumResponse other)
{
if (other ==
com.endpoint.test.Sum.SumResponse.getDefaultInstance()) return this;
if (other.hasSum()) {
setSum(other.getSum());
}
this.mergeUnknownFields(other.getUnknownFields());
return this;
}
public final boolean isInitialized() {
if (!hasSum()) {
return false;
}
return true;
}
public Builder mergeFrom(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws java.io.IOException {
com.endpoint.test.Sum.SumResponse parsedMessage = null;
try {
parsedMessage = PARSER.parsePartialFrom(input,
extensionRegistry);
} catch (com.google.protobuf.InvalidProtocolBufferException e)
{
parsedMessage = (com.endpoint.test.Sum.SumResponse)
e.getUnfinishedMessage();
throw e;
} finally {
if (parsedMessage != null) {
mergeFrom(parsedMessage);
}
}
return this;
}
private int bitField0_;
// required int64 sum = 1 [default = 0];
private long sum_ ;
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public boolean hasSum() {
return ((bitField0_ & 0x00000001) == 0x00000001);
}
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public long getSum() {
return sum_;
}
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public Builder setSum(long value) {
bitField0_ |= 0x00000001;
sum_ = value;
onChanged();
return this;
}
/**
* <code>required int64 sum = 1 [default =
0];</code>
*/
public Builder clearSum() {
bitField0_ = (bitField0_ & ~0x00000001);
sum_ = 0L;
onChanged();
return this;
}
// @@protoc_insertion_point(builder_scope:SumResponse)
}
static {
defaultInstance = new SumResponse(true);
defaultInstance.initFields();
}
// @@protoc_insertion_point(class_scope:SumResponse)
}
/**
* Protobuf service {@code SumService}
*/
public static abstract class SumService
implements com.google.protobuf.Service {
protected SumService() {}
public interface Interface {
/**
* <code>rpc getSum(.SumRequest) returns
(.SumResponse);</code>
*/
public abstract void getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request,
com.google.protobuf.RpcCallback<com.endpoint.test.Sum.SumResponse>
done);
}
public static com.google.protobuf.Service newReflectiveService(
final Interface impl) {
return new SumService() {
@java.lang.Override
public void getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request,
com.google.protobuf.RpcCallback<com.endpoint.test.Sum.SumResponse>
done) {
impl.getSum(controller, request, done);
}
};
}
public static com.google.protobuf.BlockingService
newReflectiveBlockingService(final BlockingInterface impl) {
return new com.google.protobuf.BlockingService() {
public final com.google.protobuf.Descriptors.ServiceDescriptor
getDescriptorForType() {
return getDescriptor();
}
public final com.google.protobuf.Message callBlockingMethod(
com.google.protobuf.Descriptors.MethodDescriptor method,
com.google.protobuf.RpcController controller,
com.google.protobuf.Message request)
throws com.google.protobuf.ServiceException {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.callBlockingMethod() given method descriptor for
"
"wrong service type.");
}
switch(method.getIndex()) {
case 0:
return impl.getSum(controller,
(com.endpoint.test.Sum.SumRequest)request);
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
public final com.google.protobuf.Message
getRequestPrototype(
com.google.protobuf.Descriptors.MethodDescriptor method) {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.getRequestPrototype() given method "
"descriptor for wrong service type.");
}
switch(method.getIndex()) {
case 0:
return
com.endpoint.test.Sum.SumRequest.getDefaultInstance();
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
public final com.google.protobuf.Message
getResponsePrototype(
com.google.protobuf.Descriptors.MethodDescriptor method) {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.getResponsePrototype() given method "
"descriptor for wrong service type.");
}
switch(method.getIndex()) {
case 0:
return
com.endpoint.test.Sum.SumResponse.getDefaultInstance();
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
};
}
/**
* <code>rpc getSum(.SumRequest) returns
(.SumResponse);</code>
*/
public abstract void getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request,
com.google.protobuf.RpcCallback<com.endpoint.test.Sum.SumResponse>
done);
public static final
com.google.protobuf.Descriptors.ServiceDescriptor
getDescriptor() {
return
com.endpoint.test.Sum.getDescriptor().getServices().get(0);
}
public final com.google.protobuf.Descriptors.ServiceDescriptor
getDescriptorForType() {
return getDescriptor();
}
public final void callMethod(
com.google.protobuf.Descriptors.MethodDescriptor method,
com.google.protobuf.RpcController controller,
com.google.protobuf.Message request,
com.google.protobuf.RpcCallback<
com.google.protobuf.Message> done) {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.callMethod() given method descriptor for wrong "
"service type.");
}
switch(method.getIndex()) {
case 0:
this.getSum(controller,
(com.endpoint.test.Sum.SumRequest)request,
com.google.protobuf.RpcUtil.<com.endpoint.test.Sum.SumResponse>specializeCallback(
done));
return;
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
public final com.google.protobuf.Message
getRequestPrototype(
com.google.protobuf.Descriptors.MethodDescriptor method) {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.getRequestPrototype() given method "
"descriptor for wrong service type.");
}
switch(method.getIndex()) {
case 0:
return
com.endpoint.test.Sum.SumRequest.getDefaultInstance();
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
public final com.google.protobuf.Message
getResponsePrototype(
com.google.protobuf.Descriptors.MethodDescriptor method) {
if (method.getService() != getDescriptor()) {
throw new java.lang.IllegalArgumentException(
"Service.getResponsePrototype() given method "
"descriptor for wrong service type.");
}
switch(method.getIndex()) {
case 0:
return
com.endpoint.test.Sum.SumResponse.getDefaultInstance();
default:
throw new java.lang.AssertionError("Can't get here.");
}
}
public static Stub newStub(
com.google.protobuf.RpcChannel channel) {
return new Stub(channel);
}
public static final class Stub extends
com.endpoint.test.Sum.SumService implements Interface {
private Stub(com.google.protobuf.RpcChannel channel) {
this.channel = channel;
}
private final com.google.protobuf.RpcChannel channel;
public com.google.protobuf.RpcChannel getChannel() {
return channel;
}
public void getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request,
com.google.protobuf.RpcCallback<com.endpoint.test.Sum.SumResponse>
done) {
channel.callMethod(
getDescriptor().getMethods().get(0),
controller,
request,
com.endpoint.test.Sum.SumResponse.getDefaultInstance(),
com.google.protobuf.RpcUtil.generalizeCallback(
done,
com.endpoint.test.Sum.SumResponse.class,
com.endpoint.test.Sum.SumResponse.getDefaultInstance()));
}
}
public static BlockingInterface newBlockingStub(
com.google.protobuf.BlockingRpcChannel channel) {
return new BlockingStub(channel);
}
public interface BlockingInterface {
public com.endpoint.test.Sum.SumResponse getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request)
throws com.google.protobuf.ServiceException;
}
private static final class BlockingStub implements BlockingInterface
{
private BlockingStub(com.google.protobuf.BlockingRpcChannel
channel) {
this.channel = channel;
}
private final com.google.protobuf.BlockingRpcChannel channel;
public com.endpoint.test.Sum.SumResponse getSum(
com.google.protobuf.RpcController controller,
com.endpoint.test.Sum.SumRequest request)
throws com.google.protobuf.ServiceException {
return (com.endpoint.test.Sum.SumResponse)
channel.callBlockingMethod(
getDescriptor().getMethods().get(0),
controller,
request,
com.endpoint.test.Sum.SumResponse.getDefaultInstance());
}
}
// @@protoc_insertion_point(class_scope:SumService)
}
private static com.google.protobuf.Descriptors.Descriptor
internal_static_SumRequest_descriptor;
private static
com.google.protobuf.GeneratedMessage.FieldAccessorTable
internal_static_SumRequest_fieldAccessorTable;
private static com.google.protobuf.Descriptors.Descriptor
internal_static_SumResponse_descriptor;
private static
com.google.protobuf.GeneratedMessage.FieldAccessorTable
internal_static_SumResponse_fieldAccessorTable;
public static com.google.protobuf.Descriptors.FileDescriptor
getDescriptor() {
return descriptor;
}
private static com.google.protobuf.Descriptors.FileDescriptor
descriptor;
static {
java.lang.String[] descriptorData = {
"nrsumcode.proto",nnSumRequest\022\016n\006family\030"
- "\001 \002(t\022\016n\006column\030\002
\002(t"\035n\013SumResponse\022\016n\003"
"sum\030\001 \002(\003:\001021nnSumService\022#n\006getSum\022\013.S" - "umRequest\032\014.SumResponseB n\021com.endpoint."
"testB\003SumH\001210\001\001240\001\001"
};
com.google.protobuf.Descriptors.FileDescriptor.InternalDescriptorAssigner assigner =
new com.google.protobuf.Descriptors.FileDescriptor.InternalDescriptorAssigner() {
public com.google.protobuf.ExtensionRegistry assignDescriptors(
com.google.protobuf.Descriptors.FileDescriptor root) {
descriptor = root;
internal_static_SumRequest_descriptor =
getDescriptor().getMessageTypes().get(0);
internal_static_SumRequest_fieldAccessorTable = new
com.google.protobuf.GeneratedMessage.FieldAccessorTable(
internal_static_SumRequest_descriptor,
new java.lang.String[] { "Family", "Column", });
internal_static_SumResponse_descriptor =
getDescriptor().getMessageTypes().get(1);
internal_static_SumResponse_fieldAccessorTable = new
com.google.protobuf.GeneratedMessage.FieldAccessorTable(
internal_static_SumResponse_descriptor,
new java.lang.String[] { "Sum", });
return null;
}
};
com.google.protobuf.Descriptors.FileDescriptor
.internalBuildGeneratedFileFrom(descriptorData,
new com.google.protobuf.Descriptors.FileDescriptor[] {
}, assigner);
}
// @@protoc_insertion_point(outer_class_scope)
}
2> 编写服务器端的代码
package com.endpoint.test;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.Coprocessor;
import org.apache.hadoop.hbase.CoprocessorEnvironment;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.coprocessor.CoprocessorException;
import org.apache.hadoop.hbase.coprocessor.CoprocessorService;
import
org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
import org.apache.hadoop.hbase.protobuf.ResponseConverter;
import org.apache.hadoop.hbase.regionserver.InternalScanner;
import org.apache.hadoop.hbase.util.Bytes;
import com.endpoint.test.Sum.SumRequest;
import com.endpoint.test.Sum.SumResponse;
import com.endpoint.test.Sum.SumService;
import com.google.protobuf.RpcCallback;
import com.google.protobuf.RpcController;
import com.google.protobuf.Service;
public class SumEndPoint extends SumService implements Coprocessor,CoprocessorService{
private RegionCoprocessorEnvironment env; // 定义环境
@Override
public Service getService() {
return this;
}
@Override
public void start(CoprocessorEnvironment env) throws IOException {
if (env instanceof RegionCoprocessorEnvironment) {
this.env = (RegionCoprocessorEnvironment)env;
} else {
throw new CoprocessorException("no load region");
}
}
@Override
public void stop(CoprocessorEnvironment env) throws IOException {
}
@Override
public void getSum(RpcController controller, SumRequest request,
RpcCallback<SumResponse> done) {
// 设置扫描对象
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes(request.getFamily()));
scan.addColumn(Bytes.toBytes(request.getFamily()),
Bytes.toBytes(request.getColumn()));
// 定义变量
SumResponse response = null;
InternalScanner scanner = null;
// 扫描每个region,取值后求和
try {
scanner = env.getRegion().getScanner(scan);
List<Cell> results = new ArrayList<Cell>();
boolean hasMore = false;
Long sum = 0L;
do {
hasMore = scanner.next(results);
for (Cell cell : results) {
sum = Long.parseLong(new
String(CellUtil.cloneValue(cell)));
}
results.clear();
} while (hasMore);
// 设置返回结果
response = SumResponse.newBuilder().setSum(sum).build();
} catch (IOException e) {
ResponseConverter.setControllerException(controller, e);
} finally {
if (scanner != null) {
try {
scanner.close();
} catch (IOException e) {
//e.printStackTrace();
}
}
}
// 将rpc结果返回给客户端
done.run(response);
}
}
3> 客户端测试代码
package com.endpoint.test;
import java.io.IOException;
import java.util.Map;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.coprocessor.Batch;
import org.apache.hadoop.hbase.ipc.BlockingRpcCallback;
import com.endpoint.test.Sum.SumRequest;
import com.endpoint.test.Sum.SumResponse;
import com.endpoint.test.Sum.SumService;
import com.google.protobuf.ServiceException;
public class TestClient {
public static void main(String[] args) throws Exception {
// 配置HBse
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "master,data1,data2");
conf.set("hbase.zookeeper.property.clientPort", "2181");
conf.setLong("hbase.rpc.timeout", 600000);
System.setProperty("hadoop.home.dir",
"C:/hadoopfiles/hadoop-common-2.2.0-bin-master");
// 建立一个数据库的连接
Connection conn =
ConnectionFactory.createConnection(conf);
// 获取表
HTable table = (HTable)
conn.getTable(TableName.valueOf("etable"));
long sum = 0L;
// 设置请求对象
final SumRequest request =
SumRequest.newBuilder().setFamily("cf").setColumn("value").build();
try {
// 获得返回值
Map<byte[], Long> result =
table.coprocessorService(SumService.class, null, null,
new Batch.Call<SumService, Long>() {
@Override
public Long call(SumService service) throws
IOException {
BlockingRpcCallback<SumResponse>
rpcCallback = new BlockingRpcCallback<SumResponse>();
service.getSum(null, request,
rpcCallback);
SumResponse response = (SumResponse)
rpcCallback.get();
return response.hasSum() ?
response.getSum() : 0L;
}
});
// 将返回值进行迭代相加
for (Long v : result.values()) {
sum = v;
}
// 结果输出
System.out.println("sum: " sum);
} catch (ServiceException e) {
e.printStackTrace();
}catch (Throwable e) {
e.printStackTrace();
}
table.close();
conn.close();
}
}
System.setProperty("hadoop.home.dir", "C:/hadoopfiles/hadoop-common-2.2.0-bin-master"); 这句代码是防错误用的,不具有实际意义,在hadoop-common-2.2.0-bin-master下建立bin目录放一个winutils.exe文件即可,否则会出现提示“Could not locate executable nullbinwinutils.exe in the Hadoop binaries”
此外,需要在windows下设置一下hosts文件,因为conf.set("hbase.zookeeper.quorum", "master,data1,data2");
4> 使用Endpoint协处理器
将上面的Sum类文件与用于服务端的SumEndPoint 类文件打包上传到服务器
chown hadoop:hadoop datacode.jar
chmod g w datacode.jar
先改一下权限,之后
hadoop fs -copyFromLocal sumtest.jar /input/
下面是要使用协处理器的hbase表
要将协处理器加载到这个表上
disable 'etable'
# 包名.类名|权重 com.endpoint.test.SumEndPoint|100
alter'etable',METHOD =>'table_att','coprocessor' =>'/input/sumcode.jar|com.endpoint.test.SumEndPoint|100'
enable 'etable'
包名.类名|权重 com.endpoint.test.SumEndPoint|100
# 这样也是可以的,但是在集群变换主节点的情况下,不是很好
# alter'etable',METHOD =>'table_att','coprocessor' =>'hdfs://192.168.1.215:9000/input/sumcode.jar|com.endpoint.test.SumEndPoint|100'
此外,值得注意的一点,在集群中,最好在hbase-site.xml中设置以下属性:
<property>
<name>hbase.coprocessor.abortonerror</name>
<value>false</value>
</property>
设置为false目的在于提高容错性,如果这个属性没有设置为false,则在上传的jar包存在错误的情况下,会导致表不能enable或disable,从而导致集群中的这张表无法使用,甚至会影响到其他表。
在windows中的客户端运行客户端的代码,结果如下:
2、Observer实例
这个是一个二级索引实例,即假定在initialtable表中的数据格式是这样的
row1 E 151
row2 Y 158
在向initialtable表中写入数据时,自动将以下数据写入indextable表作为二级索引,indextable第二列成为indextable的键
Y 158
1> 编写服务端代码
package com.observer.test;
import java.io.IOException;
import java.util.List;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Durability;
import org.apache.hadoop.hbase.client.HTableInterface;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.coprocessor.BaseRegionObserver;
import org.apache.hadoop.hbase.coprocessor.ObserverContext;
import
org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
import org.apache.hadoop.hbase.regionserver.wal.WALEdit;
import org.apache.hadoop.hbase.util.Bytes;
public class TestObserver extends BaseRegionObserver {
@Override
public void
postPut(ObserverContext<RegionCoprocessorEnvironment> e, Put put,
WALEdit edit, Durability durability)
throws IOException {
// indextable作为二级索引表
HTableInterface table =
e.getEnvironment().getTable(TableName.valueOf("indextable"));
// 获取值
List<Cell> cellList1 = put.get(Bytes.toBytes("cf"),
Bytes.toBytes("name"));
List<Cell> cellList2 = put.get(Bytes.toBytes("cf"),
Bytes.toBytes("value"));
// 写入数据
for (Cell cell1 : cellList1) {
// 原表的列cf:name的值作为indextable的rowkey,添加行
Put indexPut = new Put(CellUtil.cloneValue(cell1));
for (Cell cell2 : cellList2) {
// 原表的列cf:value的值作为indextable表中列cf:value的值
。
indexPut.add(Bytes.toBytes("cf"),
Bytes.toBytes("value"), CellUtil.cloneValue(cell2));
}
table.put(indexPut);
}
table.close();
}
}
2> 编写客户段代码
package com.observer.test;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
public class DataClient {
public static void main(String[] args) throws IOException {
//配置
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "master,data1,data2");
conf.set("hbase.zookeeper.property.clientPort", "2181");
//连接
Connection conn = ConnectionFactory.createConnection(conf);
HTable table = (HTable)
conn.getTable(TableName.valueOf("initialtable"));
// 写入数据
Put put = new Put(Bytes.toBytes("row01"));
put.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("name"),
Bytes.toBytes("E"));
put.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("value"),
Bytes.toBytes("151"));
table.put(put);
// 关闭资源
table.close();
conn.close();
}
}
3> 创建需要的表
4> 加载协处理器
将服务端代码打包上传集群服务器的hdfs上
chown hadoop:hadoop datacode.jar
chmod g w datacode.jar
hadoop dfs -put datacode.jar /input/
之后,将协处理器加载到初始表中
disable 'initialtable'
alter'initialtable',METHOD =>'table_att','coprocessor' =>'/input/datacode.jar|com.observer.test.TestObserver|100'
enable 'initialtable'
5> 执行客户端代码,显示结果
Hadoop HBase搭建云存储总结 PDF http://www.linuxidc.com/Linux/2013-05/83844.htm
Ubuntu Server 14.04 下 Hbase数据库安装 http://www.linuxidc.com/Linux/2016-05/131499.htm
HBase 结点之间时间不一致造成regionserver启动失败 http://www.linuxidc.com/Linux/2013-06/86655.htm
CentOS 6.3下HBase伪分布式平台搭建 http://www.linuxidc.com/Linux/2016-11/136830.htm
Hadoop集群安装&HBase实验环境搭建 http://www.linuxidc.com/Linux/2013-04/83560.htm
基于Hadoop集群的HBase集群的配置 http://www.linuxidc.com/Linux/2013-03/80815.htm‘
Hadoop安装部署笔记之-HBase完全分布模式安装 http://www.linuxidc.com/Linux/2012-12/76947.htm
CentOS 6.4下HBase集群安装 http://www.linuxidc.com/Linux/2016-11/137303.htm
HBase
的详细介绍:请点这里
HBase
的下载地址:请点这里
本文永久更新链接地址:http://www.linuxidc.com/Linux/2016-12/138718.htm
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
message AddressBook {
repeated Person person = 1;
}
在DOS该目录下运行
protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto
如
protoc -I=D:UsersggDesktopprotoc-2.6.1-win32 --java_out=D:UsersggDesktopprotoc-2.6.1-win32 D:UsersggDesktopprotoc-2.6.1-win32/addressbook.proto
后生成Java文件
// Generated by the protocol buffer compiler. DO NOT EDIT!
// source: addressbook.proto
package com.example.tutorial;
public final class AddressBookProtos {
private AddressBookProtos() {}
public static void registerAllExtensions(
com.google.protobuf.ExtensionRegistry registry) {
}
public interface PersonOrBuilder extends
// @@protoc_insertion_point(interface_extends:tutorial.Person)
com.google.protobuf.MessageOrBuilder {
/**
* <code>required string name = 1;</code>
*/
boolean hasName();
/**
* <code>required string name = 1;</code>
*/
java.lang.String getName();
/**
* <code>required string name = 1;</code>
*/
com.google.protobuf.ByteString
getNameBytes();
/**
* <code>required int32 id = 2;</code>
*/
boolean hasId();
/**
* <code>required int32 id = 2;</code>
*/
int getId();
/**
* <code>optional string email = 3;</code>
*/
boolean hasEmail();
/**
* <code>optional string email = 3;</code>
*/
java.lang.String getEmail();
/**
* <code>optional string email = 3;</code>
*/
com.google.protobuf.ByteString
getEmailBytes();
/**
* <code>repeated .tutorial.Person.PhoneNumber phone = 4;</code>
*/
java.util.List<com.example.tutorial.AddressBookProtos.Person.PhoneNumber>
getPhoneList();
/**
* <code>repeated .tutorial.Person.PhoneNumber phone = 4;</code>
*/
com.example.tutorial.AddressBookProtos.Person.PhoneNumber getPhone(int index);
/**
* <code>repeated .tutorial.Person.PhoneNumber phone = 4;</code>
*/
int getPhoneCount();
/**
* <code>repeated .tutorial.Person.PhoneNumber phone = 4;</code>
*/
java.util.List<? extends com.example.tutorial.AddressBookProtos.Person.PhoneNumberOrBuilder>
getPhoneOrBuilderList();
/**
* <code>repeated .tutorial.Person.PhoneNumber phone = 4;</code>
*/
com.example.tutorial.AddressBookProtos.Person.PhoneNumberOrBuilder getPhoneOrBuilder(
int index);
}
/**
* Protobuf type {@code tutorial.Person}
*/
public static final class Person extends
com.google.protobuf.GeneratedMessage implements
// @@protoc_insertion_point(message_implements:tutorial.Person)
PersonOrBuilder {
// Use Person.newBuilder() to construct.
private Person(com.google.protobuf.GeneratedMessage.Builder<?> builder) {
super(builder);
this.unknownFields = builder.getUnknownFields();
}
private Person(boolean noInit) { this.unknownFields = com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
private static final Person defaultInstance;
public static Person getDefaultInstance() {
return defaultInstance;
}
public Person getDefaultInstanceForType() {
return defaultInstance;
}
private final com.google.protobuf.UnknownFieldSet unknownFields;
@java.lang.Override
public final com.google.protobuf.UnknownFieldSet
getUnknownFields() {
return this.unknownFields;
}
private Person(
com.google.protobuf.CodedInputStream input,
com.google.protobuf.ExtensionRegistryLite extensionRegistry)
throws com.google.protobuf.InvalidProtocolBufferException {
initFields();
int mutable_bitField0_ = 0;
com.google.protobuf.UnknownFieldSet.Builder unknownFields =
com.google.protobuf.UnknownFieldSet.newBuilder();
try {
boolean done = false;
while (!done) {
int tag = input.readTag();
switch (tag) {
case 0:
done = true;
break;
default: {
if (!parseUnknownField(input, unknownFields,
extensionRegistry, tag)) {
done = true;
}
break;
}
case 10: {
com.google.protobuf.ByteString bs = input.readBytes();
bitField0_ |= 0x00000001;
name_ = bs;
break;
}
case 16: {
bitField0_ |= 0x00000002;
id_ = input.readInt32();
break;
}
case 26: {
com.google.protobuf.ByteString bs = input.readBytes();
bitField0_ |= 0x00000004;
email_ = bs;
break;
}
case 34: {
if (!((mutable_bitField0_ & 0x00000008) == 0x00000008)) {
phone_ = new java.util.ArrayList<com.example.tutorial.AddressBookProtos.Person.PhoneNumber>();
mutable_bitField0_ |= 0x00000008;
}
phone_.add(input.readMessage(com.example.tutorial.AddressBookProtos.Person.PhoneNumber.PARSER, extensionRegistry));
break;
}
}
}
} catch (com.google.protobuf.InvalidProtocolBufferException e) {
throw e.setUnfinishedMessage(this);
} catch (java.io.IOException e) {
throw new com.google.protobuf.InvalidProtocolBufferException(
e.getMessage()).setUnfinishedMessage(this);
} finally {
if (((mutable_bitField0_ & 0x00000008) == 0x00000008)) {
phone_ = java.util.Collections.unmodifiableList(phone_);
}
this.unknownFields = unknownFields.build();
makeExtensionsImmutable();
}
}
public static final com.google.protobuf.Descriptors.Descriptor
getDescriptor() {
return com.example.tutorial.AddressBookProtos.internal_static_tutorial_Person_descriptor;
}
protected com.google.protobuf.GeneratedMessage.FieldAccessorTable
internalGetFieldAccessorTable() {
return com.example.tutorial.AddressBookProtos.internal_static_tutorial_Person_fieldAccessorTable
.ensureFieldAccessorsInitialized(
com.example.tutorial.AddressBookProtos.Person.class, com.example.tutorial.AddressBookProtos.Person.Builder.class);
}
本文由wns9778.com发布于计算机教程,转载请注明出处:google proto buffer实战
关键词: wns9778.com