TIP: A little tutorial about String.split();

As of version 1.4, the Java API now recommends that StringTokenizer no longer be used to cut a string up, and recommends that the split() method be used instead.

This is probably a good thing for the beginners, because arrays are a thing they can get their head around. They usually know how to write a loop to scan over an array too (for(int i=0; i<array.length; i++) ), whereas the subtleties of an Enumeration are a little more difficult to grasp. (By the way, the process of using a loop like the one mentioned, to crawl over an array is actually a process of "enumerating" or "iterating", but it does not use the Java classes of Enumeration or Iterator.
This tutorial doesnt deal with Enumeration or Iterator either, so if you wish to know about them, pleas emake a separate thread)

So what is split() all about?

Very simply, it is a method you can call on a string, that causes the string to be chopped up and the bits put into an array. Unlike the StringTokenizer, it is NOT capable of returning the delimiters, so if you have a particular interest in the delimiters, dont use split.

What is a delimiter? Delimit is maybe a shorter way of saying "denoting the limit" or "indicating the edge". If you have a string like this:
hello,this,is,a,string
you could choose a comma as your delimiter. the comma indicates the end of one word and the start of the next. it would split like:
hello
this
is
a
string
Equally, you could choose the letter i to be your demiliter. In this case the split string would run as:
hello,th
s,
s,astr
ng


Split returns you a chopped up string, and as mentioned in the small text above, it chops up based on delimiters. Think of delimiters as like fence posts, and everything else is a fence panel. The only thing youre likely to be interested is the panel section, so lets take an example string:

Code:

---

string: the cat sat on the mat
fence : |===|===|===|==|===|===|

| is a fence post
= are the panel sections

---

if we split that string around the "posts" we would end up with the following, separated strings:

Code:

---

the
cat
sat
on
the
mat

---

You may wonder, what would we get if there were multiple spaces, and we were using space as a delimiter:

Code:

---

string: the    cat  sat on the  mat
fence : |===|||||===||===|==|===|||===|

---

see the multiple fence posts with nothing between them.. no panels? well you just get zero length strings out if it:

Code:

---

the
                    <-- a zero length string
                    <-- another 0-length
                    <-- third 0-length
                    <-- fourth 0-length
cat

sat
on
the


mat

---

but, there were 5 fence posts.. but only 4 blank strings? why is that? because if you hammered 5 fence posts into the ground like this:
# # # # #
_1 2 3 4

there would be 4 "somethings" between them. In this case, the 4 "somethings" are zero length strings.. and thats why theres 4 of them


so it's pretty easy, that's how strings break up, and so far nothing is really different to string tokenizer.

The Difference

there are 2 methods:

split( String )
split( String, int )

the two are the same, except that if you provide an int then the split method will stop after it made that many splits, So if you have a long string, but you know all the information you really want is in the first 7 chunks (after splitting) you can pass 7 in. You can then be sure that you get an array of chunks back that is, at most, 7 long.

remember that split is a method of a String. We call it on the string we want to split, like this:
String myString = "hello world";
myString.split( String )
myString.split( String, int )

So, unlike the string tokenizer, we dont pass in the String we want to split. The String we pass in, is the delimiter(s)..

..and here is where the real difference comes.


Delimiters

The delimiters are nothing like StringTokenizer's delimiters. With StringTokenizer you gave it a simple list:

new StringTokenizer( "hello,this.is:a;test", ",.:;")

Whenever StringTokenizer found just one of those delimiters, it would cut a chunk off the string. Very simple. You couldnt specify anything more complex than one character.

Now the delimiters are a Regular Expression.. nothing to be afraid of, just something to be aware of.

Gentle Intro to Regular Expressions

Regular Expressions are used for matching text, and if your text editor that you use to write java, is fairly capable (UltraEdit.com, TextPad.com and others), it will support RegularExpressions for finding data.
Youve probably used a find/replace dialog in your life, and if I asked you to find the next occurrence of the word "java" in a document, that would be easy.
But what if i asked you to find "a letter A, followed by any character, followed by (a number 1 or a number 2), followed by Z" ?
Wow, suddenly a lot harder..

Well Regular Expressions let you do just that. There are special symbols like:

. means "any character"
\s means "any whitespace (tab, newline, space, new paragraph etc) character"
[] are or-group markers. anything within them is OR

so that search i asked for before would be:

Code:

---

find: A.[12]Z
means: A, followed by ANY CHARACTER (the fullstop is like a wildcard),
      followed by [1 OR 2], followed by Z

it would match:
AA1Z
AB1Z
AC2Z

and hundreds more, but it wouldnt match:
aa1z
ab1z
abc1z
aa3z

---

Theres a lot in the grammar of a Regular Expression, and i wont detail it here.. there's an entire tutorial provided by Sun for that, and im thinking of writing a basic one for here, but for now you should at least be aware that split() uses a regular expression to express the delimiter. This causes some problems for the classical users of stringtokenizer..

A Problem

Suppose you wanted to split a whole document up into sentences. What separates one sentence from another? THis thing:

.

so with the new split() you could take the whole document as a String, and split it around full-stops (periods) right?

myDocument.split(".");

nope :(
Because remember, the delimtier is a regular expression, and a full stop in regex speak means "any single character", so the splitter would put fenceposts everywhere, then give you a big load of nothing!

Code:

---

text: This is my text. I will split it. Will it work?
split(".")
fence: |||||||||||||||||||||||||||||||||||||||||||||||

---

So because we want to use literal . instead of special regular expression . we must do something different..

RegEx Compilation

Regular Expressions are compiled, something like java programs are compiled. It is a formal language, and things must be written in a certain way. If you managed to write a pattern of regex that broke the language grammar, you would get a PatternSyntaxException while your program was running. One such breakage may be using "*" as a regular expression.
* means "match the preceding character zero or more times", but if you dont precede it with anything, its invalid.

There is a special character in regular expressions, which means "treat the following special character as a normal character".
That character is \

It is similar to the way you may wish to put a " character in a java string. You must put a \ before it, to let java know that the string doesnt end there:

System.out.print("to use a \" character in a string, you must put a slash \\ before it");

There, we used \" to indicate literal speech mark, and later \\ to indicate a literal slash.

If we put the pattern to split our document, as:
myDocument.split("\.");

that will make the regular expression compiler ignore the following character as a special character

BUT

the java compiler sees that \. before the regular expression compiler does, and the java compiler will try and turn \. into an 'escape character' like it does with \n \b \r \t (newline, backspace, carriage return and tab)
So to make it turn that slash, into a literal slash so that the regex compiler sees a literal slash followed by a ., we must write the following in our java source code:
myDocument.split("\\.");

\\. becomes \. (after javac) and
\. becomes "a literal full stop" (in the regular expression compiler)

super :)

note, to get a literal slash into a regex requires the regex compiler to see \\, which in turn requires you to put \\\\ into your sourcecode. \\\\ -> \\ -> \
talk about long winded.

How the RegEx is Used

In using a regular expression as a delimiter, the split() method basically starts at the start of the string and finds the first occurrence of the regular expression. The following regular expression means "a or b or c, followed by 1 or 2 or 3":
"[abc][123]"

Here is how it would split some text:


text: hey, this is a1 a little demonstra3tion of the b2 power of a rec1gulab1r expression when spc2litting text c4. it is a0 hard at a3 first, but you get used b2 it


and the split array contains:

Code:

---

hey, this is
 a little demonstr
tion of the
 power of a re
gula
r expression when sp
litting text c4. it is a0 hard at
 first, but you get used
 it

---

Notice how the strings can start with spaces too.. thats because many times the regular text occurred as a word on its own, but remember, regular expressions dont know, or care about spaces, or words, or language!just literal text!

Oops..10000 char limit hit!
[/font]

if you get a real burning for Regular Expressions, here is the full grammar of special characters

And here is the regular expression trail.. a full blown tutorial by Sun, about their regexp packages.

here is the source for a little program i wrote to test splits i was working with:

Code:

---

public class SplitTest{
  public static void main(String[] argv){
    System.out.println("Usage: java SplitTest \"string to split\" \"regex to split by\"");
   
    String[] ary = argv[0].split(argv[1]);
   
    for(int i=0;i<ary.length;i++){
      System.out.println("array["+i+"]\tlength("+ary[i].length()+")\t>"+ary[i]+"<");
    }
  }
}

---

here is the output sample (splitting on space.. in red):

Quote:

---

C:\javawork>java SplitTest " Date : 22. 4. 4 Time: 10.48 ,00009952,000.00," " "
Usage: java SplitTest "string to split" "regex to split by"
array[0] length(0) ><
array[1] length(0) ><
array[2] length(0) ><
array[3] length(0) ><
array[4] length(0) ><
array[5] length(0) ><
array[6] length(4) >Date<
array[7] length(1) >:<
array[8] length(3) >22.<
array[9] length(2) >4.<
array[10] length(1) >4<
array[11] length(5) >Time:<
array[12] length(5) >10.48<
array[13] length(0) ><
array[14] length(0) ><
array[15] length(0) ><
array[16] length(0) ><
array[17] length(0) ><
array[18] length(17) >,00009952,000.00,<

---

<attachment has been removed.. a new version that is more capable, appears later in the thread>

Heres some more advanced examples. Im trying to get this line:

Code:

---

"      Date : 22. 4. 4 Time: 10.48      ,00009952,000.00,"

---

split so that theres a small array produced of JUST the date, 22nd of the 4 of the 4 (that's 2004)

the | character means OR, not quite the same as []
[] means an or group

[abc] means "a or b or c"
[abc][123] means "a or b or c" followed by "1 or 2 or 3" (it will find a1 a2 a3 b1 b2 b3 c1 c2 c3)

But as you can see, its hard to express "(a followed by b) or (a followed by c)" with an or group. the following:
[abac] simply means "a or b or a or c"
[ab][ac] means "a or b followed by a or c"
a[ba]c means "a followed by (b or a) followed by c"
which is the same text as what im trying to express:

"(a followed by b) or (a followed by c)"
"a followed by (b or a) followed by c"

but as any mathematician will know, the brackets are a BIG DEAL

So im using | to OR the expressions.

Heres an attempt that crashed:

Quote:

---

"*Date : |\. | Time.* "
Usage: java SplitTest "string to split" "regex to split by"
Exception in thread "main" java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*Date : |\. | Time.*
^
at java.util.regex.Pattern.error(Unknown Source)
at java.util.regex.Pattern.sequence(Unknown Source)
at java.util.regex.Pattern.expr(Unknown Source)
at java.util.regex.Pattern.compile(Unknown Source)
at java.util.regex.Pattern.<init>(Unknown Source)
at java.util.regex.Pattern.compile(Unknown Source)
at java.lang.String.split(Unknown Source)
at java.lang.String.split(Unknown Source)
at SplitTest.main(SplitTest.java:5)

---

Its that "Cant use a special character than needs something before it, at the start of a line" problem i mentioned earlier. The regex:

"*Date : |\. | Time.* "

means: "zero or more occurrences of <nothing> Folloewd by D followed by a, followed by t... blah blah"

let's try again:

Quote:

---

C:\javawork>java SplitTest " Date : 22. 4. 4 Time: 10.48 ,00009952,000.00," " *Date : |\. | Time.* "
Usage: java SplitTest "string to split" "regex to split by"
array[0] length(0) ><
array[1] length(2) >22<
array[2] length(1) >4<
array[3] length(1) >4<
array[4] length(17) >,00009952,000.00,<

---

hmm.. not bad, but why is there a blank at the start?
Remember that split hammers its first fence post in at the start of the line, it then looks for a delimiter. One of my delimiters in the THIS | OR THIS | OR THIS is:

Code:

---

" *Date : " //space, star, "Date", space, colon, space

---

so way hey.. it found a delimiter starting at character 0. remember that the start of the line is conceptually before character zero:

Code:

---

H e l l o  //characters
 0 1 2 3 4  //character indexes
|          //the first fence post is before the start

---

if we then provide a delimiter that matches at character 0(e.g. "H" will match the H at position 0):

Code:

---

H e l l o  //characters
 0 1 2 3 4  //character indexes
|          //the first fence post is before the start
 |          //the second fence post is on char 0

---

whats between the first 2 posts? nothing.. thats why we are still getting a 0 length string at the first place in the array, despite providing a regex that matches right from character 0.

I tried everything I could think of, including \A meaning "start of input" and ^ meaning "start of line" but i couldnt shift the blank at the start.. i'll just have to ignore it :)

eventually I refined my regex to this:

Quote:

---

C:\javawork>java SplitTest " Date : 22. 4. 4 Time: 10.48 ,00009952,000.00," ".*Date : |\. | Time.*"
Usage: java SplitTest "string to split" "regex to split by"
array[0] length(0) ><
array[1] length(2) >22<
array[2] length(1) >4<
array[3] length(1) >4<

---

".*Date : |\. | Time.*" means:



zero or more characters, followed by the word Date, followed by a space, a colon and a space
OR
a literal character fullstop (period), followed by a space
OR
a space, followed by the word Time, followed by any number of any kind of characters

heres a colorcoded matching of where the "fence post" (delimiters) are found(ive turned the spaces into underscores just so you can see them:

Code:

---


!______Date_:_22._4._4_Time:_10.48______,00009952,000.00,

---

what i get out of the string after it is split, is shown in pink. ive also shown a ! to indicate the start of the line.. it is what becomes the 0 index in my array.

I recommend using a little app similar to SplitTest, if youre going to be using split() in any major way.. it will save you counting, and guessing where input will come, and show you actual results because you may well forget about some things, like that "start of line marker may cause array[0] to be a zero length string" thing..

Just remember that when you write your split string in on the command line, the program has already been compiled by javac, so you dont suffer the "javac.exe compiler turns \\ into \" issue.. just write your regex in using characters in exactly the same way as you see in the help file link i gave in post number 2.. So when it says "use \W to mean a word character" you literally write \W on the command line, BUT you would write \\W in a source code

Here is an updated version that will print at the end, the exact code to put in your program (It takes care of the \\ issues for you):

Code:

---


public class SplitTest{
  public static void main(String[] argv){
    System.out.println("Usage: java SplitTest \"string to split\" \"regex to split by\"");
   
    String[] ary = argv[0].split(argv[1]);
   
    for(int i=0;i<ary.length;i++){
      System.out.println("array["+i+"]\tlength("+ary[i].length()+")\t>"+ary[i]+"<");
    }
   
    String regex = argv[1];
    if(regex.indexOf("\\")>=0){
      regex = regex.replaceAll("\\\\", "\\\\\\\\");
      System.out.println("\n\nYour delimiter expression contains at least one \\ character.\n"+
                        "To prevent the java compiler interpreting this as an escape\n"+
                        "code, you should use the following split() command in your \n.java sourcecode:\n\n"+
                        "myString.split(\""+regex+"\");");
    }else{
      System.out.println("\n\nUse the following split() command in your .java sourcecode:\n\n"+
                        "myString.split(\""+regex+"\");");
    }
  }
}


---

wow, i really started something here didn't i! well done again :)

Nice work :)
Always wondered what all the fuss was about.

Hello, great tutorial and great test harness/tool :).

I took the tool and expanded on it a bit for ease of use needs, or so I hope, and I wanted to contribute back to this thread by posting it here.

It can read from the command line, it can read from a file, and is designed to work when run from Eclipse or when started from the command line (well, if you are honest at the menu selection part :P).

(I attached the whole Eclipse project in a .zip file as well as the .java file itself [which I am quoting in this post too])

Code:

---

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FilenameFilter;
import java.io.IOException;
import java.io.InputStreamReader;

public class SplitTest{

        public void splitting(String arr_s, String regex) {

                String[] ary = arr_s.split(regex);

                for(int i=0;i<ary.length;i++){
                        System.out.println("array["+i+"]\tlength("+ary[i].length()+")\t>"+ary[i]+"<");
                }

                //String regex = argv[1];
                if(regex.indexOf("\\")>=0){
                        regex = regex.replaceAll("\\\\", "\\\\\\\\");
                        System.out.println("\n\nYour delimiter expression contains at least one \\ character.\n"+
                                        "To prevent the java compiler interpreting this as an escape\n"+
                                        "code, you should use the following split() command in your \n.java sourcecode:\n\n"+
                                        "myString.split(\""+regex+"\");");
                }else{
                        System.out.println("\n\nUse the following split() command in your .java sourcecode:\n\n"+
                                        "myString.split(\""+regex+"\");");
                }

                return;

        }

        public static void main(String[] argv){
                System.out.println("Usage: java SplitTest \"string to split\" \"regex to split by\"\n");

                BufferedReader in = new BufferedReader(new InputStreamReader(System.in));

                String arr_s = null;
                String regex = null;
                String fileName = "test.txt";

                SplitTest sp = new SplitTest();
                boolean eclipse = true;

                try {       

                        boolean console = false;

                        while (true) {

                                System.out.print ("Enter [1] to input a line of text from cmd.exe/WIN32_console\n" +
                                                "      [2] to read from a file (default: test.txt)(Eclipse's console)\n" +
                                "      [3] to read from a file (default: test.txt)(cmd.exe/WIN32_console)\n>");

                                String str = in.readLine();

                                try {

                                        if (1 == Integer.valueOf(str)) {
                                                console = true;
                                                break;
                                        }

                                        else if (2 == Integer.valueOf(str)) {
                                                console = false;
                                                break;
                                        }

                                        else if (3 == Integer.valueOf(str))  {
                                                console = false;
                                                eclipse = false;
                                                break;
                                        }

                                        else {
                                                System.out.println("\nNot a valid choice, please re-try...");
                                        }
                                }

                                catch (NumberFormatException n) {

                                        System.out.println("\nNot a valid choice, please re-try...");

                                }

                        }


                        if (console) {
                                System.out.print("\nInsert the string to be split as is and press ENTER\n>");

                                arr_s = in.readLine();

                                System.out.print("\nInsert the string to be used as regexp as is and press ENTER\n>");
                                regex = in.readLine();
                                sp.splitting(arr_s, regex);

                        }

                        else {

                                System.out.print("\nInsert the string to be used as regexp as is and press ENTER\n>");
                                regex = in.readLine();

                                while (true) {
                                        try {

                                                if (null == fileName) {

                                                        fileName = in.readLine();
                                                }

                                                BufferedReader in1 = null;

                                                if (null == fileName) break;

                                                if (eclipse) in1 = new BufferedReader(new FileReader("bin\\"+fileName));

                                                else in1 = new BufferedReader(new FileReader(fileName));

                                                while (true ) {

                                                        String line1 = in1.readLine();

                                                        if (line1 == null) {
                                                                break;
                                                        }

                                                        if (line1.equals("")) continue;

                                                        System.out.println("\n\nWe are splitting: \n>" + line1 +
                                                                        "\nUsing this regular expression:\n>" +
                                                                        regex);

                                                        sp.splitting(line1, regex);

                                                }
                                                break;
                                        }

                                        catch (FileNotFoundException f) {
                                                System.out.print ("\n<!!!!> The file \"" + fileName + "\" was not found...\n" +
                                                                "\nPlease enter the desired .txt file name complete with extension \n(press CTRL+C " +
                                                "or the red button on Eclipse's console to exit this program):\n>");
                                                fileName = null;
                                        }

                                }

                        }

                }

                catch (IOException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                }
        }

}

---

I know there are sloppy parts like the menu selection one, in which you can crash the program feeding the Integer's method with a non-number... I guess I should have parsed the input better, but it was intended as a quick and easy to use tool for people who wanted to test String.split() and learn regexp... ok, I get the point :P. I guess I will have to work on it a little bit more and maybe add a GUI too :P.

Edit: fixed the NumberFormatException issue and added a nice while(true) loop to allow you to try again to use the menu correctly :).

HELP NEEDED:
i would like split a string and place it in to an array[][].
can i do that?

for example i have this string.

String a = {"cat dog rat duck owl bird fish lion"};

i would to place it in an array[][]
so that it would look like this.

array[][] = {{"cat", "dog"},
{ "rat", "duck"},
....and so on...

how do i do that?

Please start your own thread when posting a question, don't just add a post onto someone else's thread.

As to your question, the split method returns a String[] so you can't use it to directly create a 2D array. However you can use it to extract the names into a 1D array and then use whatever the required logic is to convert the 1D array into a 2D array by iterating over the array and copying the names into an appropriate 2D array.

BTW what is the logic for splitting the names into a 2D array from the example given it looks like the first 2 names go into the first element, the next 2 go into the next element etc etc.

I am a newbie to Java and to this thread too. Can someone please tell me how to split the following string.

Original string : "Hello welcome back"

I need to split this into "Hello welcome" and "back".

Thanks.....

ksp

Quote:

---

I am a newbie to Java and to this thread too.

---

So presumably you've read the previous couple of posts, and in particular the start of my reply which was: "Please start your own thread when posting a question, don't just add a post onto someone else's thread". Which begs the question why have you tacked a question on the end as well :rolleyes:

Create your own thread and I'll answer your question.