Why is StringTokenizer deprecated?

The Java documentation doesn't seem to mention anything about deprecation for StringTokenizer, yet I keep hearing about how it was deprecated long ago. Was it deprecated because it had bugs/errors, or is String.split() simply better to use overall?

I have some code that uses StringTokenizer and I am wondering if I should seriously be concerned about refactoring it to use String.split(), or whether the deprecation is purely a matter of convenience and my code is safe.

1. Java 10 String Tokenizer -- not deprecated
2. Java 9 String Tokenizer -- not deprecated
3. Java 8 String Tokenizer -- not deprecated
4. Java 7 String Tokenizer -- not deprecated
5. Java 6 String Tokenizer -- not deprecated
6. Java 5 String Tokenizer -- not deprecated

If it is not marked as deprecated, it is not going away.

From the javadoc for StringTokenizer:

>StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

If you look at String.split() and compare it to StringTokenizer, the relevant difference is that String.split() uses a regular expression, whereas StringTokenizer just uses verbatim split characters. So if I wanted to tokenize a string with more complex logic than single characters (e.g. split on \r\n), I can't use StringTokenizer but I can use String.split().

StringTokenizer is not deprecated in fact StringTokenizer is 4X faster than String.split() and in competitive programming it is used by many developers.

Source :- Faster Input for Java

There is an issue with StringTokenize ...

Split have to use regex, StringTokenizer is using String or CharSequence,

but

"a.b..".split(".") will return {"a","b",""}

and StringTokenizer of "a.b.." ... will return only {"a","b"}

And this is very tricky!!! Be Carefull!!!

Better and safer alternatives to StringTokenizer are:

Much better StrongTokenizer is in org.apache.common.lang3 ... it have much more flexibility or

com.google.common.base.Splitter

I don't think so that the reason of that is String.split method, because split is slow way to parse the string - it compiles a pattern inside.

StringTokenizer just can be replaced with a more functional classes like java.util.Scanner or your can use pattern matcher to get the groups by regexp.

1. StringTokenizer is not deprecated

2. It is little different function and output ...

For example, if you have "aaa.aa.aa" and want to split it into the parts "aaa", "aa" and "a", you can just write:

new StringTokenizer("aaa.aa.aa", ".")

If you just use:

"aaa.aa.aa".split(".")

It returns an empty array, because it matches regular expressions where . is a spacial character. So you have to escape it:

"aaa.aa.aa".split("\\.")

So basically .. split enable you to use regex ... it can be very usefull

But StringTokenizer parse text by tokens ... and token can be even special character

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

The following example shows how the String.split() method can be used to break up a string into its basic tokens:

 String[] result = "this is a test".split("\\s");

Personally I feel StringTokenizer was deprecated because it was simply an complex way of doing something pretty simple. StringTokenizer as the name implies only applied to Strings so why not just made it a method in String. Further StringTokenizer didn't support RegularExpression does not support regular expression which became extremely common in the late 90's and early '00's hence rendering it practically useless.