Concept
How can strtok() split a string into tokens?
ComputerScienceOne / Tokenizing
"Recall that tokenizing is the process of splitting up a string along some delimiter. For example, the comma delimited string, \"Smith,Joe,12345678,1985-09-08\" contains four pieces of data delimited by a comma. Our aim is to split this string up into four separate strings so that we can process each one. The C string library provides a tokenizing function:\n\nchar *strtok(char *str, const char *delim);\n\nThe first argument is the string that you want to tokenize and the second contains the delimiter that you want to split along. The second argument is actually a string and allows you to specify more than one delimiter, but we’ll restrict our attention to single character delimiters. The function returns a pointer to the first token in the string. To get the second and all subsequent tokens, we call strtok() again, but we pass it NULL as the first argument to continue parsing the same string. If we pass a new string as the first argument to strtok() the tokenization process will start over on the new string. Note that the first argument does not have the const keyword. This is because strtok() will make changes to the string during the tokenization process. If the string needs to be preserved, tokenization should be performed on a deep copy of the string. When there are no more tokens in the string, strtok() returns NULL to indicate no more tokens are in the string. This logic can be used to write a while loop to iterate over each token.\n\nConsider the following example.\n\nchar data[] = \"Smith,Joe,12345678,1985-09-08\";\n\nchar *token = NULL;\n\n// make the initial call to strtok:\n\ntoken = strtok(data, \",\");\n\nwhile(token != NULL) {\n\nprintf(\"token: %s\n\n\", token);\n\n// get the next token:\n\ntoken = strtok(NULL, \",\");\n\n}\n\nThis outputs the following:\n\ntoken: Smith\n\ntoken: Joe\n\ntoken: 12345678\n\ntoken: 1985-09-08"
Related Ideas
- How can you tokenize a String in Java?ComputerScienceOne · Tokenizing
- How can PHP split strings into tokens using explode() and preg_split()?ComputerScienceOne · Tokenizing
- How can you create an array of strings in C?ComputerScienceOne · Arrays of Strings
- How can you create static strings with character arrays?ComputerScienceOne · Character Arrays
- How can strings be modified or copied?ComputerScienceOne · Basic Operations
- How can you convert a string to a number using sscanf()?ComputerScienceOne · Conversions
- How does Java make working with strings easier?ComputerScienceOne · Basics
- What are PHP strings and how should they be treated?ComputerScienceOne · Basics