Tokenize a string: Difference between revisions
Content added Content deleted
(→{{header|MATLAB}}: works also in Octave) |
(→{{header|C++}}: replaced old code which does manual string searching with a more typical approach (getline) and a more C++-specific (ctype). Also added a boost example.) |
||
Line 289:
=={{header|C++}}==
{{works with|ANSI C++}}▼
{{
std::getline() is typically used to tokenize strings on a single-character delimiter
<lang cpp>#include <string>
#include <sstream>
#include <vector>
#include <iterator>
#include <iostream>
#include <algorithm>
std::vector<std::string> tokenize_str(const std::string & str,▼
{
std::istringstream buf(s);
for(std::string token; getline(buf, token, ','); )
v.push_back(token);
copy(v.begin(), v.end(), std::ostream_iterator<std::string>(std::cout, "."));
}</lang>
C++ allows the user to redefine what is considered whitespace. If the delimiter is whitespace, tokenization becomes effortless.
▲ vector<string> tokens;
<lang cpp>#include <string>
#include <locale>
#include <sstream>
#include <vector>
#include <iterator>
#include <iostream>
#include <algorithm>
struct comma_ws : std::ctype<char> {
static const mask* make_table() {
static std::vector<mask> v(classic_table(), classic_table() + table_size);
v[','] |= space; // comma will be classified as whitespace
return &v[0];▼
}
comma_ws(std::size_t refs = 0) : ctype<char>(make_table(), false, refs) {}
};
int main()
{
std::string s = "Hello,How,Are,You,Today";
std::istringstream buf(s);
buf.imbue(std::locale(buf.getloc(), new comma_ws));
std::istream_iterator<std::string> beg(buf), end;
copy(v.begin(), v.end(), std::ostream_iterator<std::string>(std::cout, "."));
std::cout << '\n';
}</lang>
{{works with|C++98}}
{{libheader|boost}}
The boost library has multiple options for easy tokenization.
<lang cpp>#include <string>
#include <vector>
#include <algorithm>
▲int main() {
#include <iostream>
#include <boost/tokenizer.hpp>
▲ string s("Hello,How,Are,You,Today");
int main()
{
vector<string> v(tokenize_str(s));▼
std::string s = "Hello,How,Are,You,Today";
boost::tokenizer<> tok(s);
▲ cout << v[i] << ".";
copy(v.begin(), v.end(), std::ostream_iterator<std::string>(std::cout, "."))
std::cout <<
▲ return 0;
}</lang>
|