Concept
How can you tokenize a String in Java?
ComputerScienceOne / Tokenizing
"Recall that tokenizing is the process of splitting up a string using some delimiter. For example, the comma delimited string, \"Smith,Joe,12345678,1985-09-08\" contains four pieces of data delimited by a comma. Our aim is to split this string up into four separate strings so that we can process each one. Java provides a very simple method to do this called split() that takes a string delimiter and returns an array of strings containing the tokens. For example:\n\nString data = \"Smith,Joe,12345678,1985-09-08\";\n\nString tokens[] = data.split(\",\");\n\n//tokens is { \"Smith\", \"Joe\", \"12345678\", \"1985-09-08\" }\n\nString dateTokens[] = tokens[3].split(\"-\");\n\n//dateTokens is now { \"1985\", \"09\", \"08\" }\n\nThe delimiter this method uses is actually a regular expression; a sequence of characters that define a search pattern in which special characters can be used to define complex patterns. For example, the complex expression, ^[+-]?(\\d+(\\.\\d+)?|\\.\\d+)([eE][+-]?\\d+)?$ will match any valid numerical value including scientific notation. We will not cover regular expressions in depth, but to demonstrate their usefulness, here’s an example by which you can split a string along any and all whitespace:\n\nString s = \"Alpha Beta \\t Gamma\n\nDelta \\t\n\nEpsilon\";\n\nString tokens[] = s.split(\"[\\\\s]+\");\n\n//tokens is now { \"Alpha\", \"Beta\", \"Gamma\", \"Delta\", \"Epsilon\" }"
Related Ideas
- How can PHP split strings into tokens using explode() and preg_split()?ComputerScienceOne · Tokenizing
- How can strtok() split a string into tokens?ComputerScienceOne · Tokenizing
- Why is strtok() not reentrant, and what alternative does C provide?ComputerScienceOne · Tokenizing
- How are arrays of strings used in Java?ComputerScienceOne · Arrays of Strings
- How can you create arrays and collections of strings in Java?ComputerScienceOne · Comparisons
- How does Java make working with strings easier?ComputerScienceOne · Basics
- How can you compute a substring of a string?ComputerScienceOne · Arrays of Strings
- How do you find the length of a string and iterate over its characters?ComputerScienceOne · String Methods