Faster JSON parser RapidJSON

I moved from picojson to RapidJSON which is faster and also header-only.

char buffer[target->size() + 1];
memcpy(buffer, target->c_str(), target->size());
memset(buffer + target->size(), 0, 1);  // trailing \0
cerr << boost::format("buffer:'%s'\n") % (const char*)buffer;

if (document.ParseInsitu(buffer).HasParseError()) {
  cerr << "* parse error\n";
  fprintf(stderr, "\nError(offset %u): %s\n",
          (unsigned)document.GetErrorOffset(),
          GetParseError_En(document.GetParseError()));

  return 1;
}

double hoge = document["hoge"].GetDouble();
bool page = document["page"].GetBool();
const string& piyo = document["piyo"].GetString();

const Value& ij = document["innerjson"];
StringBuffer buf;
PrettyWriter<StringBuffer> wr(buf);
ij.Accept(wr);
const char* jsIj = buf.GetString();

cout << boost::format("hoge: %f\n") % hoge;
cout << boost::format("page: %d\n") % page;
cout << boost::format("piyo: %s\n") % piyo;
cout << boost::format("innerjson: %s\n") % jsIj;

Json

{
  "hoge":23.4,
  "page":true,
  "piyo":"piyopiypo",
  "innerjson": {
    "foo": "bar",
    "baz": {
      "bar": 123,
      "bar2": "hogehoge"
    }
  }
}

Premake

I learned premake since libraries I use in my new project requires it. I found these.

  • To change C/C++ compiler, use premake.gcc.cc/cxx or CC/CXX env vars.
  • You can have ‘test’ configuration in a project, but you can’t change ‘files’ per configuration for now (will be supported in premake 4.5). -> I created ‘Test’ project.
  • OSX has a problem in -Wl,-x handling. -> I added ‘Symbols’ in all build targets.

My sample premake.

-- premake4.lua
solution "HogeSolution"
  configurations { "debug", "release" }
  -- switch gcc / clang
  --premake.gcc.cc = "gcc-5"
  --premake.gcc.cxx = "g++-5"
  premake.gcc.cc = "clang-3.6"
  premake.gcc.cxx = "clang++-3.6"

-- per configuration file list is not supported as of 2015.8
-- will be supported in premake 4.5
-- http://stackoverflow.com/questions/9158151/premake-different-excludes-command-for-each-configuration

-- A project defines one build target
  project "Hoge"

    kind "ConsoleApp"
    language "C++"
    buildoptions {
      "-std=c++11",
      "-Wall"
    }
    linkoptions {
      "-stdlib=libc++"
    }

    pchheader "pre.h"

    files {
     "**.h", "**.hpp", "**.cpp"
    }

    excludes {
      "*_test.cpp"
    }

    includedirs {
    }

    libdirs {
      "/usr/local/opt/llvm36/lib/llvm-3.6/lib"
    }

    links {
    }

    configuration "debug"
      defines { "DEBUG" }
      flags { "Symbols" }
      targetname "hoge-d"

    configuration "release"
      -- linker fails if you don't have "Symbols" on osx with -Wl,-x
      flags { "Symbols", "Optimize" }
      targetname "hoge"


  project "HogeTest"
    -- assume you have the following gmake setup
    -- 1. export REPO=your-local-repo-home
    -- 2. cd $REPO
    -- 3. git clone https://github.com/google/googletest
    -- 4. cd $REPO/googletest
    -- 5. cmake .
    -- 6. make

    -- env vars
    local repo = os.getenv("REPO")
    local gtest = repo .. "/googletest"


    kind "ConsoleApp"
    language "C++"
    buildoptions {
      "-std=c++11",
      "-Wall"
    }
    linkoptions {
      "-stdlib=libc++"
    }

    pchheader "pre.h"

    files {
      "**.h", "**.hpp", "**.cpp"
    }

    excludes {
      "main.cpp"
    }

    includedirs {
      gtest .. "/include"
    }

    libdirs {
      "/usr/local/opt/llvm36/lib/llvm-3.6/lib",
      gtest
    }

    links {
      "gtest_main",
      "gtest"
    }

    flags { "Symbols" }
    targetname "hoge-test"

How to run.

premake4 gmake

make Hoge config=debug
make Hoge config=release

make HogeTest
./hoge-test
Running main() from gtest_main.cc
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from HogeTest
[ RUN      ] HogeTest.Hoge1
[       OK ] HogeTest.Hoge1 (0 ms)
[ RUN      ] HogeTest.Hoge2
[       OK ] HogeTest.Hoge2 (0 ms)
[----------] 2 tests from HogeTest (0 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (1 ms total)
[  PASSED  ] 2 tests.

sample test:
#include "hoge.hpp"
#include "gtest/gtest.h"

class HogeTest : public ::testing::Test {
  virtual void SetUp() {}
};

TEST_F(HogeTest, Hoge1) { EXPECT_EQ(3, 3); }

TEST_F(HogeTest, Hoge2) {
  Hoge h;
  int result = h.add(-2, 2);
  EXPECT_EQ(0, result);
}

Header only CPP JSON library picojson

Tried a nice JSON library for C++ @ https://github.com/kazuho/picojson.

Json:

{
  "hoge": 12.3,
  "page": true,
  "piyo": "piyopiypo",
  "ar": [
    1,2,3,4,5
  ],
  "ar2": [
    {
      "foo": "foo string",
      "bar": 12345.6
    },
    {
      "foo": "foo2 string",
      "bar": 78.99
    }
  ]
}

Test:

#include "pre.h"
pre.h ***
#define __STDC_WANT_LIB_EXT1__ 1

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <iostream>
#include <boost/format.hpp>

#define PICOJSON_USE_INT64
#include "deps/picojson.h"

main.cpp ***
#include "pre.h"

using namespace std;

int main(int argc, char const *argv[]) {
  printf("* started.\n");

  picojson::value json;
  cin >> json;

  picojson::object& o = json.get<picojson::object>();
  int hoge = o["hoge"].get<double>();
  bool page = o["page"].get<bool>();
  string piyo = o["piyo"].get<string>();

  cout << boost::format("hoge: %d\n") % hoge;
  cout << boost::format("page: %d\n") % page;
  cout << boost::format("piyo: %s\n") % piyo;

  picojson::array ar = o["ar"].get<picojson::array>();
  for (picojson::array::iterator it = ar.begin(); it != ar.end(); it++)
  {
    cout << boost::format("ar item: %d\n") % it->get<int64_t>();
  }

  picojson::array ar2 = o["ar2"].get<picojson::array>();
  for (picojson::array::iterator it = ar2.begin(); it != ar2.end(); it++)
  {
    picojson::object& item = it->get<picojson::object>();
    cout << boost::format("foo: %s\n") % item["foo"].get<string>();
    cout << boost::format("bar: %f\n") % item["bar"].get<double>();
  }

  printf("* completed.\n");
}

Swift – C++ interop

It looks Swift can’t call C++ directly and need to call it via Objective-C++.

Swift Button Handler calls ObjectiveC class ObjCHoge’s functions as below.

@IBAction func buttonClicked(sender : AnyObject) {
    let date = NSDate()
    let formatter = NSDateFormatter()
    formatter.timeStyle = .LongStyle
    
    let a:Int32 = 10
    let b:Int32 = 20
    let hoge = ObjCHoge()
    
    var str = String(format: "[%@] %d+%d=%d\n", formatter.stringFromDate(date), a, b, hoge.add(a,b))
    insertMessage(str)
    str = String(format: "[%@] %d-%d=%d\n", formatter.stringFromDate(date), a, b, hoge.sub(a,b))
    insertMessage(str)
}

BridgingHeader.h should have pure Objective-C++ header which I think can’t have C++ classes.

#import "ObjCHoge.h"

Then ObjCHoge.h/.mm calls CppHoge methods. I didn’t want to use “void*” for storing CppHoge instance, but it looked ObjCHoge.h should be pure Objective-C++ header and cannot have CHoge* for Swift to call.
Hm… Objective-C (OSX/iOS UI)/C# (Win UI)/Java (Android UI) – C++ (logic) is more handy than Swift (OSX/iOS UI)/C# (Win UI)/Java (Android UI) – C++ (logic) when writing cross platform code for OSX/iOS.

Let me know if there is a better way.

#import <Foundation/Foundation.h>

@interface ObjCHoge : NSObject
{
    // it looks ObjCHoge.h should be pure Objective-C for Swift to call
    // and can't define CppHoge* _cppHoge
    void* _cppHoge;
}

-(int)add:(int)a :(int)b;
-(int)sub:(int)a :(int)b;
-(int)static_add:(int)a :(int)b;
-(int)static_sub:(int)a :(int)b;

@end

#include "ObjCHoge.h"
#include "CppHoge.h"

@implementation ObjCHoge
-(id)init{
    self = [super init];
    
    if (self) {
        _cppHoge = new CppHoge();
    }
    
    return self;
}

-(void)dealloc {
    delete static_cast<CppHoge*>(_cppHoge);
}

-(int)add:(int)a :(int)b {
    CppHoge& obj = *static_cast<CppHoge*>(_cppHoge);
    return obj.add(a, b);
}

-(int)sub:(int)a :(int)b {
    CppHoge& obj = *static_cast<CppHoge*>(_cppHoge);
    return obj.sub(a, b);
}

-(int)static_add:(int)a :(int)b {
    return CppHoge::static_add(a, b);
}

-(int)static_sub:(int)a :(int)b {
    return CppHoge::static_sub(a, b);
}
@end

CppHoge.h/.cpp

//
//  CppHoge.h
//

#ifndef __cp_osx__CppHoge__
#define __cp_osx__CppHoge__

#include <stdio.h>

class CppHoge
{
public:
    int add(int a, int b);
    int sub(int a, int b);
    static int static_add(int a, int b);
    static int static_sub(int a, int b);
};

#endif /* defined(__cp_osx__CppHoge__) */

//
//  CppHoge.cpp
//

#include "CppHoge.h"


int CppHoge::add(int a, int b){
    return a+b;
}

int CppHoge::sub(int a, int b){
    return a-b;
}

int CppHoge::static_add(int a, int b){
    return a+b;
}

int CppHoge::static_sub(int a, int b){
    return a-b;
}

My uncrustify config

Diff from the default config for c++. Good parameter definition.

> input_tab_size                           = 4        # number
> output_tab_size                          = 4        # number
> indent_columns                           = 4        # number
> indent_with_tabs                         = 0        # number
> indent_class                             = true     # false/true
> indent_class_colon                       = true     # false/true
> nl_fdef_brace                            = force    # ignore/add/remove/force

before:

#include "global.h"

#ifndef __c2dxspike__Stone__
#define __c2dxspike__Stone__
#include "HelloWorldScene.h"
#include <iostream>

typedef enum tagState {
kStateGrabbed,
kStateUngrabbed
} touchState;

class Stone : public CCNode, CCTargetedTouchDelegate {
public:
Stone();
virtual ~Stone();
...

CCRect Stone::setStoneColor(int stoneType)
{
this->stoneType = stoneType;
CCRect color;

switch (this->stoneType) {
case 0:
color = CCRectMake(68, 2, 32, 32);
break;

case 1:
color = CCRectMake(132, 2, 32, 32);
break;

case 2:
color = CCRectMake(2, 2, 32, 32);
break;

case 3:
color = CCRectMake(100, 2, 32, 32);
break;
}

return color;
}

after uncrustify *.cpp *.h –no-backup:

#include "global.h"

#ifndef __c2dxspike__Stone__
#define __c2dxspike__Stone__
#include "HelloWorldScene.h"
#include <iostream>

typedef enum tagState {
    kStateGrabbed,
    kStateUngrabbed
} touchState;

class Stone : public CCNode, CCTargetedTouchDelegate {
public:
    Stone();
    virtual ~Stone();
...

CCRect Stone::setStoneColor(int stoneType)
{
    this->stoneType = stoneType;
    CCRect color;

    switch (this->stoneType) {
    case 0:
        color = CCRectMake(68, 2, 32, 32);
        break;

    case 1:
        color = CCRectMake(132, 2, 32, 32);
        break;

    case 2:
        color = CCRectMake(2, 2, 32, 32);
        break;

    case 3:
        color = CCRectMake(100, 2, 32, 32);
        break;
    }

    return color;
}

Cocos2d-x Xcode config for iOS / Andriod

    1. Install Xcode, Android SDK/NDK
    2. Craete cocos 2d-x project in Xcode
    3. Update NDK_ROOT_LOCAL and ANDROID_SDK_ROOT in create-android-project.sh
    4. Run create-android-project.sh
    5. Copy generated proj.android folder into the iOS project root
    6. Copy platform/android/java/src/org/cocos2dx/lib/* into proj.android/src/org/cocos2dx/lib/*
    7. Update COCOS2DX_ROOT in build_native.sh
    8. Use Eclipse, or create a new target for Android in the Xcode project … I created a new target as ‘external build system’ which calls build_android_ver.sh.
#!/bin/sh

function test_ret_code(){
▸   ret=$?
▸   if [ $ret -ne 0 ]; then
▸   ▸   echo "error: $1 failed, ret:$ret"
▸   ▸   exit 1
▸   fi
}

export  NDK_ROOT="/Users/sokoide/android-ndk-r8b"
echo "*** custom build started"
/Users/sokoide/Projects/iPhone/Spike/c2dxspike/c2dxspike/proj.android/build_native.sh
test_ret_code "build_native.sh"
/usr/bin/ant -f /Users/sokoide/Projects/iPhone/Spike/c2dxspike/c2dxspike/proj.android/build.xml debug install
test_ret_code "install"
/Users/sokoide/android-sdks/platform-tools/adb shell am start -n com.sokoide.c2dx/.spike
test_ret_code "adb shell am start"

echo "*** custom build completed"
exit 0

cocos2dx on iOS and Android Test app on iOS (480×320) and Android (800×480). I scaled the background, but didn’t for the label on top.

C++ 11 summary video by Embarcadero Japan

Watched C++11 1/2 and C++11 2/2, and tried some of them. It was quick & nice summary.

void Hoge::LambdaTest()
{
    int v1=42;
    auto f1 = [v1] { return v1; };  // capture value
    auto f2 = [&v1] { return v1; }; // capture reference
    auto f3 = [=] { return v1; };   // capture all values
    
    v1 = 99;
    auto f1result = f1();
    auto f2result = f2();
    
    cout << "f1: " << f1result << endl;
    cout << "f2: " << f2result << endl;
    cout << "f3: " << f3() << endl;
}

void Hoge::PointerTest(){
    auto sp1 = make_shared<int>(42);    //shared_ptr<int> sp1(new int(42));
    auto sp2 = sp1;

    if (sp1) { cout << "sp1: " << *sp1 << endl; }
    if (sp2) { cout << "sp2: " << *sp2 << endl; }
    
    unique_ptr<string> up1(new string("hoge"));
    // unique_ptr<string> up2 = up1;
    unique_ptr<string> up2(up1.release());
    
    if (up1) { cout << "up1: " << *up1 << endl; } // up1 alread released
    if (up2) { cout << "up2: " << *up2 << endl; }
}


void Hoge::InitTest(){
    //int *p1 = new int[10];      // new only
    //int *p2 = new int[10]();    // new + clear
    
    const int elements = 10;
    string* p3 = new string[elements]{"ab", "cd", "ef"};
    int* p4 = new int[elements]{1,2};
    
    cout << "string:" << endl;
    for (int i=0; i<elements; i++){
        cout << i << "[string] :" << p3[i] << endl;
        cout << i << "[int] :" << p4[i] << endl;
            }
}

void Hoge::FuncTest(){
    function<int(int, int)> f = [](int i, int j){ return i*j;};
    cout << f(1,2) << endl;
    cout << f(3,4) << endl;
}

void Hoge::TupleTest(){
    tuple<string, double> book1("B1 123-456", 42.0);    // constructor
    tuple<string, double> book2{"B2 123-456", 42.1};    // initializer
    auto book3 = make_tuple("B3 123-456", 42.2);        // returns <const char*, double>
    auto book4 = make_tuple(string("B4 123-456"), 42.3);// returns <string, double>
    
    cout << "book1: " << get<0>(book1) << ", " << get<1>(book1) << endl;
    cout << "book2: " << get<0>(book2) << ", " << get<1>(book2) << endl;
    cout << "book3: " << get<0>(book3) << ", " << get<1>(book3) << endl;
    cout << "book4: " << get<0>(book4) << ", " << get<1>(book4) << endl;
}

void Hoge::RandomTest(){
    // unsigned int
    default_random_engine e1((unsigned int)time(0));  // unsigned int
    for(int i=0; i<10; i++) {
        cout << "e1:" << e1() << endl;
    }
    
    uniform_int_distribution<unsigned long> u(0, 9);
    for(int i=0; i<10; i++) {
        cout << "u(e1):" << u(e1) << endl;
    }
    
    // 64bit
    mt19937_64 e2(time(0));  // unsigned long long (64bit)
    for(int i=0; i<10; i++) {
        cout << "e2:" << e2() << endl;
    }
}

Thread Synchronization in C++11

1. Tried condition_variable to signal from one to another. Something like one of sync objects (event, mutex, critial section, …) and WaitForSingleObject() in Windows. Lambda in the cond.wait() was called twice. How should I remove the 1st call by creating unsignaled cond?

code:

mutex mtx;
condition_variable cond;
bool done = false;

thread t = thread([&]() {
	// define _GLIBCXX_USE_NANOSLEEP
		this_thread::sleep_for(chrono::seconds(2));
		cout << "[" << this_thread::get_id() << "] notifying" << endl;
		done = true;
		cond.notify_one();
	});

thread t2 = thread([&]() {
	cout << "[" << this_thread::get_id() << "] waiting..." << endl;
	unique_lock<mutex> lock(mtx);
	cond.wait(lock, [&]() {
				cout << "[" << this_thread::get_id() << "] notified. done=" << done << endl;
				return done == true;
			});
});

t.join();
t2.join();
cout << "all done" << endl;

output:

[0x105bcb000] waiting...
[0x105bcb000] notified. done=0
[0x105b48000] notifying
[0x105bcb000] notified. done=1
all done

2. Borrowed rendezvous from here to suspend all threads until specified number of threads call wait(), then resume running them.

code:

const int numThreads = 10;
rendezvous r(numThreads + 1);
mutex mtx;

vector<thread> threads(numThreads);
for (int i : range(0, numThreads)) {
	threads[i] = thread([&]() {
		this_thread::sleep_for(chrono::seconds(2));
		{
			lock_guard<mutex> guard(mtx); // just to synchronize print output
			cout << "[" << this_thread::get_id() << "] done" << endl;
		}
		r.wait();
	});
}

r.wait();
cout << "all done" << endl;
for (int i : range(0, numThreads)) {
	threads[i].join();
}

output:

[0x10a83c000] done
[0x10a942000] done
[0x10a8bf000] done
[0x10a9c5000] done
[0x10aa48000] done
[0x10aacb000] done
[0x10ab4e000] done
[0x10abd1000] done
[0x10a736000] done
[0x10a7b9000] done
all done

Thread in C++11

Yesterday’s bug marks got almost disappeared by adding -std=c++11 in Preprocessor Include Path,

However it’s not perfect. For example, thread::join() should be valid and there is no error mark at ‘t.join()’. But if you store the handle in vector element t2, it says ‘Method “join” could not be resolved” although it’s valid 🙁

Anyway, I tried C++11 threads. I borrowed range from here.

void thread_test() {
	const int numThreads = 10;
	mutex mtx;

    vector<thread> threads(numThreads);
    for(int i:range(0,numThreads)){
    	threads[i] = thread([&mtx]() {
    		lock_guard<mutex> guard(mtx);
    		cout << "[" << this_thread::get_id() << "] Hoge!" << endl;
    	});
    }

    for(int i:range(0,numThreads)){
    	threads[i].join();
    }
}

Enabling C++11

I got a clang LLVM 1.0 error when compiling C++ Lambda with Xcode 4.3.3. It seems Xcode 4.3 doesn’t support it although the latest clang supports it. So, I installed gcc4.7 which supports it.

sudo port install gcc_select
sudo port install gcc47
sudo port select gcc mp-gcc47

It worked in a terminal, but didn’t work well with Eclipse which needed the following settings.

1. Right click on your project -> properties -> C/C++ build -> Environment -> add "PATH = /opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:".
2. project -> properties -> C/C++ Build -> Tool Chain Editor -> Choose Linux GCC.
3. project -> properties -> C/C++ Build -> Settings -> Tool Settings -> GCC C++ Compiler -> Miscellaneous -> Tool Sttings -> add "-std=c++11"


One more thing, but I don’t know how to resolve the following false positive errors although compile succeeds.

Finally I can experiment C++11 features. I began with Lambda, auto, range-based for loop.

void lamda_test()
{
   auto f = [](){};
   f(); // do nothing

   int ar[] = {1, 2, 3, 4, 5};
   auto f2 = [=]() { for(auto& i:ar) { cout << i << endl;} };	// capture ar (value)
   f2(); // print 1,2,3,4,5

   vector<int> v;
   for (int i = 0; i < 10; i++) {
       v.push_back(i);
   }
   int sum = 0;
   for_each(v.begin(), v.end(), [&](int x) { sum += x; }); // capture sum by '&'(reference)
   cout << sum << endl; // print 45
}
1 2