Kafka Confluent Platform
Confluent is a company founded by the team that built Apache Kafka. It builds a platform around Kafka that enables companies to easily access data as real-time streams.
Confluent offers three different ways to get started with Kafka.
- Confluent Open Source
- Confluent Enterprise
- Confluent Cloud
While we in this series of Kafka Tutorial discuss much about Confluent Open Source, you may check the other two ways based on your requirement and interest.
While comparing Confluent Open Source with Apache Kafka, we get the following capabilities or tools in addition to standard Apache Kafka :
- Additional Clients : Supports C, C++, Python, .NET and several other non-Java Clients.
- REST Proxy – Provides universal access to Kafka from any network connected device via HTTP
- Schema Registry – Central registry for the format of Kafka data – guarantees all data is always consumable
- Pre-Built Connectors – HDFS, JDBC, Elasticsearch, Amazon S3 and other connectors fully certified and supported by Confluent
To install Confluent Open Source on Ubuntu, following is a step by step guide :
1. Install Confluent public key
$ wget -qO – http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add –
root@tutorialkart:/home/arjun# wget -qO - http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add - OK
2. Add the repository to sources list
$ sudo add-apt-repository “deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main”
root@tutorialkart:/home/arjun# sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main"
3. Update Packages
$ sudo apt-get update
4. Install Confluent Open Source Platform
$ sudo apt-get install confluent-platform-oss-2.11
2.11 at the end refers to the scala version that is currently supported.
root@tutorialkart:/home/arjun# sudo apt-get install confluent-platform-oss-2.11 Reading package lists... Done Building dependency tree Reading state information... Done ...
Following would be the locations
|/usr/bin/||Confluent CLI and individual driver scripts for starting/stopping services, prefixed with <package> names|
|/etc/<package>/||Configuration files. <package> : [confluent-common/ confluent-control-center/ confluent-control-center-fe/ confluent-rebalancer/]|
|/usr/share/java/<package>/||Jars <package> : [kafka/ kafka-connect-jdbc/ kafka-connect-storage-common/ kafka-connect-elasticsearch/ kafka-connect-replicator/ kafka-rest/ kafka-connect-hdfs/ kafka-connect-s3/ kafka-serde-tools/ ]|
5. Start Confluent
You may start all or some of the services using confluent command line interface with start command.
root@tutorialkart:~# confluent start Starting zookeeper zookeeper is [UP] Starting kafka kafka is [UP] Starting schema-registry schema-registry is [UP] Starting kafka-rest kafka-rest is [UP] Starting connect connect is [UP]
In this Kafka Tutorial, we have learnt about Confluent Platform and how to install Confluent Open Source Platform.