In order to achieve a predefined confidence level interval I need to collect simulation results data from many runs (each with a different seed). What is the recommended way to collect the data?

In order to achieve a predefined confidence level interval I need to collect simulation results data from many runs (each with a different seed). What is the recommended way to collect the data?

The way to go here is to collect the results of each run in an output scalar file. The scalar statistics collected in such a single run are associated with the seed used. Values written to an output scalar file are appended each time new data is collected; this is different than output vector files, which are overwritten with each simulation run of a scenario.To write the seed to the scalar file, you need to use the KP op_stat_scalar_write(). Typically you will write exactly one value in the scalar file per statistic per simulation.To gather the confidence interval, you have to be able to plot your scalar file results in a manner that for each point in the X axis, there are several points on the Y axis (which were gathered over several simulation runs). However, if you plot your statistic (be it delay, throughput, max queue size or anything else) against the simulation seed, each simulation will have a different X-axis location.Determine your X-axis simulation attribute (such as offered load), and for each value of this attribute, you have to run several discrete-event simulations with different seeds. Write the X-axis attribute to the output scalar file, either via an attribute probe or by using op_stat_scalar_write() in your model code.As a result, after running N simulations for K values of the X-axis attributes, you will get NxK points on the resulting scalar graph. For every X-axis point, N values may be used to calculate the confidence interval.Note that the question How many simulations do I have to run to reach a certain interval? is invalid. If your results have inherently large standard deviation, no matter how many simulation are run, the interval will remain large. What you gain by running many simulations is improved accuracy of the confidence interval: when you represent the results of the confidence intervals, you must state the ALPHA probability (tail probability you used to calculate the interval), AND the number of samples N (same N as before) you used.For more information on the subject of confidence intervals, see...- FAQ 913, as well as FAQs with the keyword scalarModeler documentation:- vol. Simulation Kernel, ch. Statistic Package.- vol. Modeling Concepts, ch. Data Analysis, sec. Computing Confidence in Simulation Results- vol. Modeling Concepts, ch. Modeling and Simulation Basics, sec. Discrete Event Simulation Issues, subsec. Developing Confidence in Simulation Results- vol. Editor Reference, ch. Project Editor, sec. Pop-Up Menu Operations, subsec. Edit Graph Properties- vol. Editor Reference, ch. Analysis Tool