Saturday, July 19, 2014

Benchmarking Part 2 - Creating and Destroying Entities


My previous post identified three processes that can bottleneck a typical simulation model:
  1. Event scheduling and processing.
  2. Creating and destroying entities.
  3. Executing model blocks that do not involve simulated time.
Each of these activities has the potential to be slow enough to bottleneck a model's execution speed. By measuring the processing times for these activities, we can assess the relative strengths and weaknesses for each simulation software package.

Event scheduling and processing was investigated in my previous post. I now move on to the second item on the list: creating and destroying entities.

Entity Creation and Destruction Benchmark

The simulation model used to benchmark entity creation and destruction is simplicity itself. An EntityGenerator is used to create a series of entities that are sent immediately to an EntitySink block for destruction.  The following figure shows the JaamSim model for this benchmark.

Benchmark Model
The average time to create and destroy an entity was measured by running the model for 60 seconds of real time (using a stopwatch) and counting the number of entities that were destroyed. The effect of the computer speed was allowed for by converting the calculated time into clock cycles. All measurements were made using my laptop computer which has a second generation (Sandybridge) Core i5 processor running at 2.5 GHz.

Performance Results

The following bar chart shows the results of the benchmarking exercise for Arena, Simio, and for the same three versions of JaamSim that were benchmarked in the previous post.

The first thing to note about these results is that event creation and destruction is a very time consuming process! Even for the fastest version of JaamSim, the entity creation/destruction time is about fifteen times longer than the time to execute a delay. This activity can easily be the bottleneck for a simulation model that processes large numbers of entities.

The version of JaamSim initially tested, JaamSim2014-11, was quite slow due to the slow method for event scheduling and processing used in this version (one future event is needed to generate an entity at a future time). Once a faster method was used in JaamSim2014-16, its time was only slightly slower than Arena and more than twice as fast as Simio. For the final version tested, JaamSim2014-20, some simple optimizations were performed that reduced the processing time by a factor of three to "only" 23,000 clock cycles.

Concluding Remarks

Although JaamSim is now significantly faster, there is still room for further improvement. The time to create and destroy an entity is about fifteen times longer than the time to execute a time delay.

The slowness of event creation/destruction is significant only when very large numbers of entities are to be created and destroyed. On the laptop computer used for these benchmarks, JaamSim can create and destroy about 110,000 entities per second. At this rate, only 9 seconds of processing time are required for a million entities - an insignificant amount. However, one hundred million would require 15 minutes and one billion would require 2.5 hours. Simulations involving such large number of entities will require more efficient entity creation and destruction methods or a model design that pools and recycles the entities instead of destroying them.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.