The input line is too long when starting kafka

JavaApache Kafka

Java Problem Overview

I'm trying to run Kafka message queue on Windows.

I'm usin this tutorial - https://dzone.com/articles/running-apache-kafka-on-windows-os

When i try to run it with comand - .\bin\windows\kafka-server-start.bat .\config\server.properties

and i get an error: The input line is too long. The syntax of the command is incorrect.

kafka location - C:\kafka_2.11-1.0.0

Java Solutions

Solution 1 - Java

This is because of the long length of the path because of the folder name 'kafka_2.11-1.0.0'. Just rename the folder to something small, like just 'kafka'.

Solution 2 - Java

The Problem

The kafka-run-class.bat file performs a bunch of CLASSPATH :concat calls that make the CLASSPATH very long.

Depending on your environment, too long: Windows cmd.exe environment has a limit of 8191 characters.

Solutions

Edit kafka-run-class.bat so that ...

  1. make it so that CLASSPATH is not used or set elsewhere
  2. make paths shorter so that concat produces a string smaller than than 8191 characters
  3. make concat use the whole folder instead of every single jar (via libs/*)

Example

Here is an example of an edit to kafka-run-class.bat (source) that uses the 2nd approach:

replace ...

rem Classpath addition for release
for %%i in ("%BASE_DIR%\libs\*") do (
	call :concat "%%i"
)

... by this ...

rem Classpath addition for release
call :concat "%BASE_DIR%\libs\*;"

Solution 3 - Java

Just moving the Kafka path to a much smaller root path solved the issue. For e.g. "C:\Kafka"

Solution 4 - Java

Above both options didn't work for me.

I have just moved an unzipped directory to C:/ drive and started power shell in Administrator mode and tried the desired commands, the zookeeper and broker started smoothly.

Solution 5 - Java

I have copied everything to C:/tools/kafka and it works fine. The problem was indeed the too long path to kafka.

Solution 6 - Java

For me, it worked only after keeping the Kafka folder right under C directory, so the path looked like C:\Kafka. I am sure this will work.

Solution 7 - Java

Windows command-line has issues with long command execution. Use Powershell instead.

Solution 8 - Java

put all the files in other drive like D: or E: and run the command

Solution 9 - Java

Place kafka close to the root of your drive so that the path to it is very short.

When you run those Kafka batch files included in the windows directory, they muck with your environment variables (the classpath one) and can create a very long input line to actually run the command/jar.

Also as some others pointed out, ensure you downloaded the binary (yes even though on the Kafka site it is called the Scala binary and there's no mention of Windows anywhere, which can confuse people), not the source code. Also confusing to the lay-user, the source distribution looks similar to the binary distribution when uncompressed, it has all the batch files for example but they won't run.

Solution 10 - Java

The input line is too long when starting kafka

I tried below command but gives me same error

C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

This happens because path gets append in classpath when we are continuously executing same command on same cmd.

Close your command prompt and open once again and then start zookeeper server.

Solution 11 - Java

For Windows just keep Kafka Folder path short For eg - D/dev/kafka/bin/windows

Solution 12 - Java

Closing Zookeeper and Kafka cmd window and reopening it again worked for me. I tried all options above except powershell.

Solution 13 - Java

Rename the kafka folder to some shorter name like kafka2. Open a new command prompt (DO not use existing/opened command prompt)

Solution 14 - Java

I tried extracting the downloaded Kafka zip file contacts

C:\Kafka and ran the Kafka-server-start.bat

from the command prompt (don't run the command prompt as administrator) and that worked.

Running the command prompt as administrator was causing the problem for me.

Solution 15 - Java

Remove kafka_2.13-2.5.0 if there's any and keep the folder name as simple as possible like Kafka, also make sure there are no spaces in the path.

A working example: Folder is located in C:\kafka.
A not working example: Folder is located in C:\kafka 2\.

Solution 16 - Java

I had this problem too. Debugging the bat-files, you can see, that changing into the directory before calling the command that returns with this error and calling it with relative path does not afect the result! The problem is the CLASSPATH variable. The easiest way to get this shorter for me was to use the command:

subst K: <absolute-long-path-to-kafka-dir-with-version>

This way I could change into this virtual drive cd /d K: and execute the scripts without problem:

K:\bin\windows\kafka-server-start.bat K:\config\server.properties

I wrote a wrapper-script that removes the CLASSPATH variable before executing again, because the values are appended every execution (same cmd.exe).

set CLASSPATH=
cd /d K:
K:\bin\windows\kafka-server-start.bat K:\config\server.properties

Solution 17 - Java

I'm using scoop. Changing ...bin\windows\kafka-run-class.bat as below (kudos Midiman) fixed it for me:

@REM rem Classpath addition for kafka-core dependencies
@REM for %%i in ("%BASE_DIR%\core\build\dependant-libs-%SCALA_VERSION%\*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\core\build\dependant-libs-%SCALA_VERSION%\*"

@REM rem Classpath addition for kafka-examples
@REM for %%i in ("%BASE_DIR%\examples\build\libs\kafka-examples*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\examples\build\libs\*"

@REM rem Classpath addition for kafka-clients
@REM for %%i in ("%BASE_DIR%\clients\build\libs\kafka-clients*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\clients\build\libs\*"

@REM rem Classpath addition for kafka-streams
@REM for %%i in ("%BASE_DIR%\streams\build\libs\kafka-streams*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\streams\build\libs\*"

@REM rem Classpath addition for kafka-streams-examples
@REM for %%i in ("%BASE_DIR%\streams\examples\build\libs\kafka-streams-examples*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\streams\examples\build\libs\*"

@REM for %%i in ("%BASE_DIR%\streams\build\dependant-libs-%SCALA_VERSION%\rocksdb*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\streams\build\dependant-libs-%SCALA_VERSION%\*"

@REM rem Classpath addition for kafka tools
@REM for %%i in ("%BASE_DIR%\tools\build\libs\kafka-tools*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\tools\build\libs\*"

@REM for %%i in ("%BASE_DIR%\tools\build\dependant-libs-%SCALA_VERSION%\*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\tools\build\dependant-libs-%SCALA_VERSION%\*"

for %%p in (api runtime file json tools) do (
	@REM for %%i in ("%BASE_DIR%\connect\%%p\build\libs\connect-%%p*.jar") do (
	@REM 	call :concat "%%i"
	@REM )
	call :concat "%BASE_DIR%\connect\%%p\build\libs\*"
	if exist "%BASE_DIR%\connect\%%p\build\dependant-libs\*" (
		call :concat "%BASE_DIR%\connect\%%p\build\dependant-libs\*"
	)
)

@REM rem Classpath addition for release
@REM for %%i in ("%BASE_DIR%\libs\*") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\libs\*"

@REM rem Classpath addition for core
@REM for %%i in ("%BASE_DIR%\core\build\libs\kafka_%SCALA_BINARY_VERSION%*.jar") do (
@REM 	call :concat "%%i"
@REM )
call :concat "%BASE_DIR%\core\build\libs\*"

Solution 18 - Java

Rename kafka_2.11-1.0.0 to kafka and keep it in C drive. This worked for me.

Solution 19 - Java

Please check the directory path of your files, it should not contain any "-" in your directory structure where you have kept Kafka. Changing the name of the directory from kafka_2.13-2.7.0 --> Kafka worked for me.

Solution 20 - Java

Another good way to solve this is to create a virtual drive from the kafka folder:

subst <drive:> path\to\folder

example:

subst K: C:\Users\me\software\kafkax.x-x.x

You can then go to the K: drive and run zookeeper and kafka without problem.

cd K:
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
.\bin\windows\kafka-server-start.bat .\config\server.properties

Solution 21 - Java

Run common .\bin\windows\kafka-server-start.bat .\config\server.propertie from Powershell window, it works without any issue on Window.

For a similar error, for zookeeper I run the zookeeper zkserver.cmd which i downloaded from https://zookeeper.apache.org/doc/r3.6.2/zookeeperStarted.html

Categories
Recommended Java Solutions
Recommended Apache Kafka Solutions

Previous Solution:

create typeface from font resource id

AndroidFontsAndroid CollapsingtoolbarlayoutAndroid Typeface

Next Solution:

Are nested structured bindings possible?

C++C++17Structured Bindings

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionGehaView Question on Stackoverflow
Solution 1 - JavaPrashant_MView Answer on Stackoverflow
Solution 2 - JavaMidimanView Answer on Stackoverflow
Solution 3 - JavaAnandView Answer on Stackoverflow
Solution 4 - JavadrtView Answer on Stackoverflow
Solution 5 - JavaAdam BodrogiView Answer on Stackoverflow
Solution 6 - JavaShridhar KulkarniView Answer on Stackoverflow
Solution 7 - JavaLukas MaruniakView Answer on Stackoverflow
Solution 8 - JavaYESHWANTH LNView Answer on Stackoverflow
Solution 9 - JavaJohn FantasticoView Answer on Stackoverflow
Solution 10 - JavaPriyanka BauskarView Answer on Stackoverflow
Solution 11 - JavaGautam MishraView Answer on Stackoverflow
Solution 12 - JavaSonal ChauhanView Answer on Stackoverflow
Solution 13 - Javauser1416932View Answer on Stackoverflow
Solution 14 - JavaKarView Answer on Stackoverflow
Solution 15 - JavaOshin ChandelView Answer on Stackoverflow
Solution 16 - JavaMayra DelgadoView Answer on Stackoverflow
Solution 17 - JavaDonlon E. McGovernView Answer on Stackoverflow
Solution 18 - JavaPrashant KanakView Answer on Stackoverflow
Solution 19 - JavaMayur ChavanView Answer on Stackoverflow
Solution 20 - JavaMissingSemiColonView Answer on Stackoverflow
Solution 21 - JavakeeplearningView Answer on Stackoverflow