我有一组在Thrift中定义的structs
,例如以下内容:
struct Foo {
1: i32 a,
2: i64 b
}
我需要用C++做以下操作:
(a) 将
Foo
的实例序列化为Thrift兼容的字节(可以使用Binary
或Compact
Thrift协议)。(b) 将字节序列化的实例发送到一个
Kafka
主题。问题:
我如何将
Thrift
序列化的实例发送到Kafka
集群?提前感谢。
我有一组在Thrift中定义的structs
,例如以下内容:
struct Foo {
1: i32 a,
2: i64 b
}
Foo
的实例序列化为Thrift兼容的字节(可以使用Binary
或Compact
Thrift协议)。Kafka
主题。Thrift
序列化的实例发送到Kafka
集群?我自己找到了问题的答案。
下面的代码片段演示了如何将 Foo
的实例序列化为 Thrift
兼容字节(使用Thrift Compact
协议)。
要使用 Binary
协议,请将 TCompactProtocol
替换为 TBinaryProtocol
。
#include <thrift/transport/TBufferTransports.h>
#include <thrift/protocol/TCompactProtocol.h>
using apache::thrift::protocol::TCompactProtocol;
using apache::thrift::transport::TMemoryBuffer;
...
...
boost::shared_ptr<TMemoryBuffer> buffer(new TMemoryBuffer());
boost::shared_ptr<TCompactProtocol> protocol(new TCompactProtocol(buffer));
uint8_t **serialized_bytes = reinterpret_cast<uint8_t **>(malloc(sizeof(uint8_t *)));
uint32_t num_bytes = 0;
// 'foo' is an instance of Foo
foo->write(protocol.get());
buffer->getBuffer(serialized_bytes, &num_bytes);
以下代码片段展示了如何将Thrift兼容的字节发送到Kafka集群。
注意: 下面使用的kafka客户端库是librdkafka。
#include "rdkafkacpp.h"
std::string errstr;
// Create global configuration
RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL);
conf->set("metadata.broker.list", "localhost:9092", errstr);
conf->set("api.version.request", "true", errstr);
// Create kafka producer
RdKafka::Producer *producer = RdKafka::Producer::create(conf, errstr);
// Create topic-specific configuration
RdKafka::Topic *topic = RdKafka::Topic::create(producer, "topic_name", nullptr, errstr);
auto partition = 1;
// Sending the serialized bytes to Kafka cluster
auto res = producer->produce(
topic, partition,
RdKafka::Producer::RK_MSG_COPY /* Copy payload */,
serialized_bytes, num_bytes,
NULL, NULL);
if (res != RdKafka::ERR_NO_ERROR) {
std::cerr << "Failed to publish message" << RdKafka::err2str(res) << std::endl;
} else {
std::cout << "Published message of " << num_bytes << " bytes" << std::endl;
}
producer->flush(10000);