STRING TOKENIZER IN JAVA

The string tokenizer class allows an application to split strings into tokens. The tokenization method is musch simpler than one used by the StreamTokenizer class. The StringTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

The default delimiters are –

1. The space character.

2. The tab character (\t)

3. The new line character (\n)

4. The carriage-return character (\r)

5. The form-feed character (\f)

From version 1.4 onwards it is recommended to use string class “split” method rather than StringTokenizer class.

Constructors of StringTokenizer class –

1. public StringTokenizer(String str) – It breaks the given string with above default delimiters.

2. public StringTokenizer (String str, String delim) – It breaks the given string with given delimiter.

3. public StringTokenizer(String str, String delim, Boolean returndelims) – It breaks the given string with given delimiter and also includes delimiter as token.

Methods of StringTokenizer-

1. public int countToken() – To get token count.

2. public boolean hasMoreElements() – To check is token available or not.

3. public boolean hasNextToken() – To check is token available or not.

4. public Object nextElement() – To retrieve token

5. public String nextToken() - To retrieve token

Example –

import java.util.StringTokenizer;

public class Tokenizer {

          public static void main(String[] args) {

                   StringTokenizer st = new StringTokenizer("I love coding in java");

                   System.out.println(st.countTokens());

                   while(st.hasMoreTokens())

                   {

                             System.out.println(st.nextToken());

                   }

                   System.out.println();

                   StringTokenizer st1 = new StringTokenizer("i love coding in java", "o");

                   while(st1.hasMoreTokens())

                   {

                             System.out.println(st1.nextToken());

                   }

                   System.out.println();

                   StringTokenizer st2 = new StringTokenizer("i love coding in java", "o", true);

                   System.out.println(st2.countTokens());

                   System.out.println();

                   while(st2.hasMoreTokens())

                   {

                             System.out.println(st2.nextToken());

                   }

                   System.out.println();

                   StringTokenizer st3 = new StringTokenizer("i love coding in java", "o", false);

                   System.out.println(st3.countTokens());

                   System.out.println();

                   while(st3.hasMoreTokens())

                   {

                             System.out.println(st3.nextToken());

                   }

          }

}

Output –

5

I

love

coding

in

java

i l

ve c

ding in java

5

i l

o

ve c

o

ding in java

3

i l

ve c

ding in java