Kafka Confluent Platform
Confluent is a company founded by the team that built Apache Kafka. It builds a platform around Kafka that enables companies to easily access data as real-time streams.
Confluent offers three different ways to get started with Kafka.
- Confluent Open Source
- Confluent Enterprise
- Confluent Cloud
While we in this series of Kafka Tutorial discuss much about Confluent Open Source, you may check the other two ways based on your requirement and interest.
While comparing Confluent Open Source with Apache Kafka, we get the following capabilities or tools in addition to standard Apache Kafka :
- Additional Clients : Supports C, C++, Python, .NET and several other non-Java Clients.
- REST Proxy – Provides universal access to Kafka from any network connected device via HTTP
- Schema Registry – Central registry for the format of Kafka data – guarantees all data is always consumable
- Pre-Built Connectors – HDFS, JDBC, Elasticsearch, Amazon S3 and other connectors fully certified and supported by Confluent
To install Confluent Open Source on Ubuntu, following is a step by step guide :
Install Confluent public key
$ wget -qO – http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add –12root@tutorialkart:/home/arjun# wget -qO - http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add -OK
Add the repository to sources list
$ sudo add-apt-repository “deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main”1root@tutorialkart:/home/arjun# sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main"
$ sudo apt-get update
Install Confluent Open Source Platform
$ sudo apt-get installconfluent-platform-oss-2.11
2.11 at the end refers to the scala version that is currently supported.12345root@tutorialkart:/home/arjun# sudo apt-get install confluent-platform-oss-2.11Reading package lists... DoneBuilding dependency treeReading state information... Done...
Following would be the locations
/usr/bin/ Confluent CLI and individual driver scripts for starting/stopping services, prefixed with <package> names /etc/<package>/ Configuration files. <package> : [confluent-common/ confluent-control-center/ confluent-control-center-fe/ confluent-rebalancer/] /usr/share/java/<package>/ Jars <package> : [kafka/ kafka-connect-jdbc/ kafka-connect-storage-common/
kafka-connect-elasticsearch/ kafka-connect-replicator/ kafka-rest/
kafka-connect-hdfs/ kafka-connect-s3/ kafka-serde-tools/ ]
You may start all or some of the services using confluent command line interface with start command.Start Confluent Services1234567891011root@tutorialkart:~# confluent startStarting zookeeperzookeeper is [UP]Starting kafkakafka is [UP]Starting schema-registryschema-registry is [UP]Starting kafka-restkafka-rest is [UP]Starting connectconnect is [UP]
In this Kafka Tutorial, we have learnt about Confluent Platform and how to install Confluent Open Source Platform.