Performance Measurements

I did some timing tests on various aspects of the NoPtr library:

• Constructing/destructing free DAO's
• Constructing/destructing DAO's in a container
• Returning an unnamed temporary DAO from a function
• Accessing the DAO: create one DAO and access it a large number of times;
• Re-ordering a container of DAO's: populate a container, then do a sort and random-shuffle a large number of times.

I compared these operations with the exact same ones if the proxies were replaced by raw pointers. Each of the four tests is done by a different file, located in the NoPtrLibRootDir/tests/performance (NoPtrLibRootDir is wherever you put the NoPtr lib). So far the timing results are great, except for container re-ordering. The table lists the time (in seconds) for executing every test, for raw pointers, for the NoPtr equivalents, and for the boost::shared_ptr equivalent, and the hit is for NoPtr vs raw:

    ====================================================================
    |       |  Construct  Return_tmp  Access  Construct_STL  Re-order  |
    --------------------------------------------------------------------
    | raw   |     21         21         27         20           13     |
    | noptr |     22         25         28         31          113     |
    | boost |     48         56         28         39          113     |
    --------------------------------------------------------------------
    | % hit |      5         18          3         60          770     |
    ====================================================================
    

Discussion:

• Both access and construct have hits at or under 5%, well within acceptable limits for most applications;
• Returning a temporary from a function is a bit higher hit, almost 20%. In a typical program, where sources and sinks of ownership tend to be localized (for simpler program logic), this operation will be a fairly rare compared to other tasks of the program;
• Construct within STL containers is a disappointing 60% hit. This could be improved to 20% by using NoPtr::InValueContainerOpt2 as the implementation of the value-based container context. Again, typical program having DAO's that require containers of pointers will be for DAO's that are expensive to create; here the DAO is an int so the hit in a normal program may be substancially smaller;
• Re-order: Sorting + random_shuffling takes almost nine times longer than for bare DAO's. This one took me by surprise, so I replaced DynObj with boost::shared_ptr and re-did this test. boost::shared_ptr takes exactly the same amount of time, so though surprising, the factor of 9 seems to be correct. Most likely if the std::sort and std::random_shuffle made better use of std::swap this factor would go down tremendously for both boost::shared_ptr and NoPtr::DynObj.

So other than for re-ordering the elements of an STL container via std:: algorithms, the performance hit for having the functionality of NoPtr should be minimal for the typical program. Hopefully, the above numbers will help you estimate what would be the hit for your particular application and if the extra functionality of NoPtr is worth it.

Todo:

do a vector re-ordering test for RRef

figure out if re-ordering performance can be improved for DynObj

Author:

Oliver Schoenborn

Since:

Apr 2003

---