Productizing Human Behavior Models

I was alerted to the article The Sims Meet Science, by Franz Dill's great blog on emerging technology. The application domain here is the movement of people through space, for urban and architectural planning, but many of the issues are common to all of the agent-based modeling work I've been involved in...

Complexity

A big part of making the simulations more accurate lies in increasing the intelligence of individual agents representing pedestrians. Xiaolin Hu, assistant professor at Georgia State University, said... The engineering challenge lies in figuring out how to balance complexity, accuracy, and performance. To create more accuracy, researchers want to make the decision-making model of the agents more complex, but this affects performance. Hu said, "You may end up where you add all this extra computation on the decision-making part, but don't gain much from the results point of view."

Calibration

Aside from not adding anything useful to the results, extra complexity introduced to bring a model closer to reality creates more parameters that need to be calibrated, and obtaining the data for calibration is the achilles heel of any behavioral model:

When you need to know how a pedestrian space will work in a particular part of the world, you need to have the measurement data from that part of the world. Otherwise, how can you have confidence in your simulations?" explained Kevin Mannion, CEO of Legion.

Presentation

One of the challenges in building a business around agent-based modeling is, as with any new technology, customer adoption. If a model is going to influence the behavior of decision-makers, they need to have confidence in the model's accuracy. A prerequisite to building this confidence is engaging the decision-makers though an experience of the model. This helps make the abstract concept of mathematical modeling more concrete, building a bridge to the everyday real world:

One of the more exciting trends combines pedestrian simulation with 3D animation. More illustrative and realistic uses of pedestrian simulation could help improve the architectural process by showing a decision maker how people move through a building. [Alex] Schmid [managing director of Savannah Simulations] explained, "First and foremost, these are engineering tools designed for runtime analysis. The graphics are added on after the simulation to make it easier to sell a concept or new design alternative."

Standards

Another challenge in selling modeling is being able to build models at a cost that provides a profit relative to the customers perception of the value. One thing that would help is standardization in the underlying simulation platform, so that domain experts can focus on modeling the domain:

At the moment, all commercial pedestrian-simulation packages are proprietary and run on a Windows PC. But researchers are looking at ways to abstract away the different modeling system levels to give modelers the same freedom that computer operating systems give programmers across different hardware implementations.

Just imagine if every time a business wanted to do something with their data, they had to write their own database software! That's where we are today with simulation. The challenge, in my experience, with standardized platforms for agent-based modeling is that they don't provide enough leverage to be worth using. Handling simulation events, while not trivial, is a small part of the problem. It's like saying you need to be able to save files to implement a database.

A much larger issue is to find an effective way to represent behaviors, modify them, and make them clearly observable to others who have not developed the model. The database analogy would be graphical data modeling tools.

There are some intermediate steps which hold great potential in building the use of agent-based models, and human behavior simulation models in general: Connecting together the models that exist today. I'll discuss that in another post.