Building Kafka 0.8.0 from sources on Windows

Prerequisites – Java JDK

Although Kafka is written in Scala, you do not need to install it. Only Java JDK is required.

Download Java JDK:
http://www.oracle.com/technetwork/java/javase/downloads/index.html

Install it and make sure you update the PATH environment variable according to instructions:
http://docs.oracle.com/javase/7/docs/webnotes/install/windows/jdk-installation-windows.html

Get the source

Do either the Clone-and-Checkout step or the Download step:

• Clone-and-Checkout
  Execute:

  git clone http://git-wip-us.apache.org/repos/asf/kafka.git kafka
  cd kafka
  git checkout -b 0.8 remotes/origin/0.8

• Download
  Download from https://github.com/apache/kafka ("Download Zip" button) and unpack to the folder kafka. Execute:

  cd kafka

Build

Execute:

.\sbt.bat update
.\sbt.bat package
.\sbt.bat assembly-package-dependency

Update the bat files

Unfortunately the bat files for Kafka 0.8.0 are full of errors, so in order to start Zookeeper and Kafka they must be replaced. See Step 1, Update the bat files.

Update config for Windows

Update the config files according to Step 1, Update Config.

Run

To run, see Step 2, Start the server.

 

See Running Kafka 0.8.0 on Windows for more info on how to set up and run Kafka using the binary distribution.

Kafka 0.8.0 on Windows

Getting Kafka 0.8.0 running on Windows isn’t straight forward if you follow the instructions. They are somewhat misleading, and the bat files are old. But with correct instructions and updated bat files it’s easy and can be done under 10 minutes. Some say you need Cygwin in order to run Kafka. This is not true. Only Server JRE is required.

Step 0. Prerequisite – Java SE Server JRE

You need Java SE Server JRE in order to run Kafka. If you have JDK installed, you already have Server JRE installed.

1. Download Java SE Server JRE
   http://www.oracle.com/technetwork/java/javase/downloads/index.html
   For me Chrome changed the extension. If that happens change it back to .tar.gz in order to unpack it.
2. Unpack it to a folder, for example c:\JreServer
   Update the system environment variable PATH to include C:\JreServer\jre\bin (Control Panel and search for environment variable).

Step 1. Download Kafka

1. Download the binaries for Kafka from http://kafka.apache.org/downloads.html
2. Unzip to a folder, for example c:\kafka

Update the bat files

Unfortunately the bat files for Kafka 0.8.0 are full of errors, so in order to start Zookeeper and Kafka they must be replaced.

1. Download updated windows bat files from https://github.com/HCanber/kafka/releases
2. Unzip and copy them into c:\kafka\bin\windows (overwrite the files already there)

Update Config

The config files need to be updated.

1. Open config\server.properties and locate log.dirs=/tmp/kafka-logs. If you keep the default it will result in an error later on. Set it to a full path without . or .. in it and with forward slashes. Example:

   log.dirs=c:/kafka/kafka-logs

2. This step is optional but you might want to set the data directory for Zookeeper as well. Open config\zookeeper.properties and locate dataDir=/tmp/zookeeper. Example:

   dataDir=c:/kafka/zookeeper-data

Step 2. Start the server

Note! Before executing the commands below you need to change directory to the folder where you unzipped Kafka:

cd c:\kafka

1. Start Zookeeper in a Command Prompt or PowerShell Console.

   .\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

2. Start Kafka in another Command Prompt or PowerShell Console.

   .\bin\windows\kafka-server-start.bat .\config\server.properties

Step 3. Create a topic

1. Create a topic.

   .\bin\windows\kafka-create-topic.bat --zookeeper localhost:2181 --replica 1 --partition 1 --topic test

2. List topics.

   .\bin\windows\kafka-list-topic.bat --zookeeper localhost:2181

Step 4. Send some messages

1. Start Console Producer

   .\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test

2. Write some messages

   This is a message 
   This is another message

Step 5. Start a consumer

1. Start Console Consumer

   .\bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic test --from-beginning

 

Building from sources

See Building Kafka 0.8.0 from sources on Windows