Social simulation: what criticism do we get?

This week on the SIMSOC listserv was a request from Annie Waldherr & Nanda Wijermans for modellers of social systems to complete a short questionnaire on the sort of criticism they receive. The questionnaire is only two short questions, one asking what field you are in and the other asking you to 'Describe the criticism you receive. For instance, recall the questions or objections you got during a talk you gave. Feel free to address several points.'

Here was my quick response to the second question:

1) Too many 'parameters' in agent-based models (ABM) make them difficult to analyse rigorously and fully appreciate the uncertainty of (although I think this kind of statement highlights the mis-understanding some have of how ABM can be structured - often models of this type are more reliant on rules of interactions between agents than individual parameters).

2) The results of models are seen as being driven by the assumptions of the modeller than by the state of the real world. That is, modellers may learn a lot about their models but not much about the real world (see similar point made by Grimm [1999] in Ecological Modelling 115)

I think it would have been nice to have a third question offering an opportunity to suggest how we can, or should, respond to these critisisms. Here's what I would have written if that third question was there:

To address point 1) above we need to make sure that we:

i) document our models comprehensively (e.g., via ODD) so that others understand model structure and can identify likely important parameters/rules and assumptions;

ii) show that the model parameter space has been widley explored (e.g., via use of techniques like Latin hypercube sampling).

To address 2) we need to make sure that:

iii) when documenting our models (see i) we fully justify the rationale of our models, hopefully with reference to real world data;

iv) we acknowledge and emphasise that the current state of ABM means that usually they can be no more than metaphors or sophisticated analogies for the real world but that they are useful for providing alternative means to think about social phenomena (i.e., they have heuristic properties).

If you're working in this area go and share your thoughts by completing the short questionnaire , or leaving comments below.