I need a combination of strtok and strtok_single

Question

I have the following line that I am trying to parse for variables.

char data[]=" to=[e-mail] & cc=[e-mail] &title=&content=how are you?&signature=best regards." 

I started with strtok and the following code

 char *to=parsePostData("to",data); char* parsePostData(char s[],char t[]) { char *postVal; char *pch; char tCpy[512];//Make a copy. Otherwise, strtok works on the char pointer, and original char array gets modified/ corrupted. strcpy(tCpy,t); pch = strtok (tCpy,"=&"); while (pch != NULL) { if(strcmp(pch,s)==0) { pch= strtok (NULL, "&"); return pch; }else{ pch = strtok (NULL, "=&"); } } } 

This works fine except when it comes to consecutive delimiters such as "title". So I found this regular strtok_single implementation. You need to know when there is no data between two token separators using strtok ()

 char * strtok_single (char * str, char const * delims) { static char * src = NULL; char * p, * ret = 0; if (str != NULL) src = str; if (src == NULL) return NULL; if ((p = strpbrk (src, delims)) != NULL) { *p = 0; ret = src; src = ++p; } return ret; } 

But the problem with this is that I can’t get the “signature”, because after that there is no separator either.

How can I get a combination of these two, so I will not miss the last variable, and I can handle consecutive delimiters?

Answer 1

There are two errors here. One is in strtok_single() . If you run it several times, it does not return the last segment after = after the signature, unlike strtok() .

When this is fixed, there is still a problem with the code in parsePostData() ; it returns a pointer to an automatic variable. Copy the string to be processed in different ways; the easiest way (which is consistent with using strtok() rather than strtok_r() or strtok_s() ) is to make the tCpy variable static.

emt.c testing emt.c

This is a complex program that shows problems, as well as a set of fixes. It applies different "splitter" functions to the data - functions with the same signature as strtok() . It demonstrates an error in strtok_single() and that strtok_fixed() fixes this error. It demonstrates that the code in parsePostData() works correctly when it is fixed, and strtok_fixed() .

 #include <stdio.h> #include <string.h> /* Function pointer for strtok, strtok_single, strtok_fixed */ typedef char *(*Splitter)(char *str, const char *delims); /* strtok_single - as quoted in SO 30294129 (from SO 8705844) */ static char *strtok_single(char *str, char const *delims) { static char *src = NULL; char *p, *ret = 0; if (str != NULL) src = str; if (src == NULL) return NULL; if ((p = strpbrk(src, delims)) != NULL) { *p = 0; ret = src; src = ++p; } return ret; } /* strtok_fixed - fixed variation of strtok_single */ static char *strtok_fixed(char *str, char const *delims) { static char *src = NULL; char *p, *ret = 0; if (str != NULL) src = str; if (src == NULL || *src == '\0') // Fix 1 return NULL; ret = src; // Fix 2 if ((p = strpbrk(src, delims)) != NULL) { *p = 0; //ret = src; // Unnecessary src = ++p; } else src += strlen(src); return ret; } /* Raw test of splitter functions */ static void parsePostData1(const char *s, const char *t, Splitter splitter) { static char tCpy[512]; strcpy(tCpy, t); char *pch = splitter(tCpy, "=&"); while (pch != NULL) { printf(" [%s]\n", pch); if (strcmp(pch, s) == 0) printf("matches %s\n", s); pch = splitter(NULL, "=&"); } } /* Fixed version of parsePostData() from SO 30294129 */ static char *parsePostData2(const char *s, const char *t, Splitter splitter) { static char tCpy[512]; strcpy(tCpy, t); char *pch = splitter(tCpy, "=&"); while (pch != NULL) { if (strcmp(pch, s) == 0) { pch = splitter(NULL, "&"); return pch; } else { pch = splitter(NULL, "=&"); } } return NULL; } /* Composite test program */ int main(void) { char data[] = " to=[e-mail] & cc=[e-mail] &title=&content=how are you?&signature=best regards."; char *tags[] = { "to", "cc", "title", "content", "signature" }; enum { NUM_TAGS = sizeof(tags) / sizeof(tags[0]) }; printf("\nCompare variants on strtok()\n"); { int i = NUM_TAGS - 1; printf("strtok():\n"); parsePostData1(tags[i], data, strtok); printf("strtok_single():\n"); parsePostData1(tags[i], data, strtok_single); printf("strtok_fixed():\n"); parsePostData1(tags[i], data, strtok_fixed); } printf("\nCompare variants on strtok()\n"); for (int i = 0; i < NUM_TAGS; i++) { char *value1 = parsePostData2(tags[i], data, strtok); printf("strtok: [%s] = [%s]\n", tags[i], value1); char *value2 = parsePostData2(tags[i], data, strtok_single); printf("single: [%s] = [%s]\n", tags[i], value2); char *value3 = parsePostData2(tags[i], data, strtok_fixed); printf("fixed: [%s] = [%s]\n", tags[i], value3); } return 0; } 

Example output from emt

 Compare variants on strtok() strtok(): [to] [ [e-mail] ] [cc] [ [e-mail] ] [title] [content] [how are you?] [signature] matches signature [best regards.] strtok_single(): [to] [ [e-mail] ] [cc] [ [e-mail] ] [title] [] [content] [how are you?] [signature] matches signature strtok_fixed(): [to] [ [e-mail] ] [cc] [ [e-mail] ] [title] [] [content] [how are you?] [signature] matches signature [best regards.] 

and

 Compare variants on strtok() ✓ strtok: [to] = [ [e-mail] ] ✓ single: [to] = [ [e-mail] ] ✓ fixed: [to] = [ [e-mail] ] ✓ strtok: [cc] = [ [e-mail] ] ✓ single: [cc] = [ [e-mail] ] ✓ fixed: [cc] = [ [e-mail] ] ✕ strtok: [title] = [content=how are you?] ✓ single: [title] = [] ✓ fixed: [title] = [] ✓ strtok: [content] = [how are you?] ✓ single: [content] = [how are you?] ✓ fixed: [content] = [how are you?] ✓ strtok: [signature] = [best regards.] ✕ single: [signature] = [(null)] ✓ fixed: [signature] = [best regards.] 

Correct (✓ = U + 2713) and incorrect labels (✕ = U + 2715) were added manually when sending a response.

Please note that only tags marked as “fixed” contain exactly what is required each time.

Answer 2

You definitely did not tell us what you mean by “it works great”, although it seems sufficient to say that you want to parse the string application/x-www-form-urlencoded . Why didn’t you say it first?

Note that the first key field may be interrupted by the first of '=' or '&' . It would be advisable to look for a token that ends with one of these characters in order to extract the key .

The second value field, however, does not end with the '=' character, so it is inconvenient to look for this character to retrieve the value . You want to search only '&' .

Of course. You can use strtok to parse this, but I'm sure there are many more suitable tools. strcspn , for example, will not make any changes to data , which means you won’t need to make a copy of data like you ...

 #include <stdio.h> #include <string.h> int main(void) { char data[]=" to=[e-mail] & cc=[e-mail] &title=&content=how are you?&signature=best regards."; char *key = data; do { int key_length = strcspn(key, "&="); char *value = key + key_length + (key[key_length] == '='); int value_length = strcspn(value, "&"); printf("Key: %.*s\n" "Value: %.*s\n\n", key_length, key, value_length, value); key = value + value_length + (value[value_length] == '&'); } while (*key); return 0; }