IL-2/SEOW Performance

[IV/JG7_4Shades]

From Petr at the HQ Forum:

Salute!
In our SEOW matches at the 69.giap we seem to run into problems whenever we seem to cross a certain barier with regard to ground objects and moves per turn. The current stable limit is about 30 ground moves per turn per side. Go over this and we experience crashes and disconnects. The disconnects seem to be random as often the players suffering from this have excellent internet connections and pings and very advanced hardware gaming machines.

We are using the 64 seat mod but the most players we have had online at the same time will be around 45. We've had problems happen on the Crimea, Stalingrad and Indonesia map among others. We do have production/Factories enabled on the Crimea map.

The hardware we are using is a Dual core intell Processor, 2GB Ram running XP 32-bit. The connection we have is 10Mbit up and down. The server is also running TS and the SEOW software is installed on this machine. However, the SQL and Web services are stopped during mission time.

The version of IL2 is 4.09, HSFX 3.2(not sure about version but it's the latest)

We are currently not using supply or trains and other advanced features. The more knowledgebale members of the squad associated these features with more crashes and so discourage the use of these features.

So my question, what hardware are you guys using to run the zillion of units I see for the campaigns being run. Take for instance the Case Blue multi sector campaign being proposed in another topic, what hardware would one need to run one of the map missions?

Are there any tips on how to configure IL2/SEOW to limit the load on the server? Anything you know on how to push up that limit for ground unit moves would really be appreciated! Thanks in advance!

SEOW rocks!
Salute
=69.GIAP=Petr

and my answer:

Hi Petr,

Thanks for your post. Designing an SEOW campaign is not easy, especially if you are trying to optimize frame rates. I'll give some general guidelines below, but please note that here, at SEOW HQ, we typically do not run campaigns with large numbers of pilots. For large campaigns, this thread would best be placed at the Official SEOW Forum, where it will be read by many more people. If you look at the statistics of the Taifun campaign we ran recently, all those missions were run on machine less powerful than your server and with supply on.

Some ways to optimize SEOW performance:

1/ Use only a small number of types of objects. The less variety of objects in your campaign, the easier for IL-2 to allocate memory to them all.

2/ Limit engagement ranges of artillery pieces.

3/ Turn off wreckage objects.

4/ Limit aircraft loadouts so there are less pauses whilst the game loads new explosions/effects.

5/ Decrease the parking percentage to reduce the number of static planes on the ground.

6/ Increase the vehicle breakdown rate to reduce the number of active artillery guns.

7/ Choose a season where clouds are limited, e.g. summer, to help frame rates.

8/ Place infrastructure targets only by GATTACK directives to reduce dummy target objects.

9/ Place front line markers only via strategic control.

10/ Do not initialize the campaign with units at full strength.

11/ Disable automatic artillery emplacements and limit the use of scenery objects.

12/ Combat ships cost more FPS than ground units.

If you have a good example of a mission file that crashes, send it to me. I can tell you pretty quickly if I can see anything notable or wrong in there.

Cheers,
4Shades

[Jager]

All players should lower there graphics settings also a bit from normal for SEOW missions, that could help on some disconnections.

Also make sure everybody have the same network setting on il2, 56 or isdn.

Do your ground/sea/supply/recon movements on different hour then the flying mission hour.
This shows you how much load the ground units create to your missions.

Isn't the historymod a lot more heavier to run then the basic il2 4.09?
Disable all the eyecandy mods from it maybe?

[=gRiJ=Petr]

Hi guys,

I've been taking some time to get more experience with the seow software and I've been building a template for a Normandy campaign I'd like to do. I'm really aiming for historical accuracy and the campaign has fixed starting setups, and Order of battles with units arriving at historical times and places. With all starting units in the template I have some 420 odd ground moves alone.

This is a side effect of having to move all mobile units at least 2 waypoints. However, if I select stationary armour and artillery for mobile flak and such, the game accepts it to. Would it be a problem to import this template? Would SEOW automatically "top up" the armour units to 4?

For the moment, our testing hardware seems to be ok, however, loading the mission planner webpage takes approx. 6-10 minutes if you take the global sector or a sector with lots of units. We are using the latest version of everything linked to the mysql software. The question: would performace increase if we were to use SQL 2005? I have the software and license so that is not an issue.

If the answer above is yes, which steps do I need to take to link SEOW to the SQL software/DB?

One more question, in your campaigns, how many total objects do you have at maximum, approximately? How many ground and naval moves do you have per turn? How many human pilots are participating and how many AI flights?

Thanks again for you help!

If other members with experience would post their settings and hardware/bandwith etc... here that would be fantastic! This info is so difficult to find out there and I think it would give new guys a much needed insight in what is needed to get a campaign going on the hardware and settings front!!

Hi Petr,

Thanks for your post. Designing an SEOW campaign is not easy, especially if you are trying to optimize frame rates. I'll give some general guidelines below, but please note that here, at SEOW HQ, we typically do not run campaigns with large numbers of pilots. For large campaigns, this thread would best be placed at the Official SEOW Forum, where it will be read by many more people. If you look at the statistics of the Taifun campaign we ran recently, all those missions were run on machine less powerful than your server and with supply on.

Some ways to optimize SEOW performance:

1/ Use only a small number of types of objects. The less variety of objects in your campaign, the easier for IL-2 to allocate memory to them all.

Hmmm, rgr but what would be a good balance? How many different unit types would be reasonable?
Also, Does the amount of memory available on the machine help IL2? Or does it only take a limited amount which is hardcoded?

2/ Limit engagement ranges of artillery pieces.

There is a problem with this. My testing has shown that heavy flak units have long ranges. For example the 88mm Flak 18 has 10000+ range while heavy arty like 152mm is limited to 6000m. Since heavy flak also fires at ground targets this may cause a balancing problem. What is the consensus on this? Any recommendations by the veterans of what keeps it balanced?

4/ Limit aircraft loadouts so there are less pauses whilst the game loads new explosions/effects.

hmmm, ok, what do you recommend? Any harline not to cross?

5/ Decrease the parking percentage to reduce the number of static planes on the ground.

Which percentage seems to be the sweet spot? 25%? 50%

6/ Increase the vehicle breakdown rate to reduce the number of active artillery guns.

What does this vehicle breakdown rate actually do? Are these vehicles lost permantly?

8/ Place infrastructure targets only by GATTACK directives to reduce dummy target objects.

I don't understand what you mean. Could you explain? Thanks!

10/ Do not initialize the campaign with units at full strength.

Hmmm, again, not sure what you mean. Could you explain?

If you have a good example of a mission file that crashes, send it to me. I can tell you pretty quickly if I can see anything notable or wrong in there.

Cheers,
4Shades

PS: Can I copy this post to SEOW Official Forum?

Best,
Petr

[=gRiJ=Petr]

All players should lower there graphics settings also a bit from normal for SEOW missions, that could help on some disconnections.

Also make sure everybody have the same network setting on il2, 56 or isdn.

Do your ground/sea/supply/recon movements on different hour then the flying mission hour.
This shows you how much load the ground units create to your missions.

Isn't the historymod a lot more heavier to run then the basic il2 4.09?
Disable all the eyecandy mods from it maybe?

Hi Jager,
Thanks for your advice. We'll try disabling all the mods we don't want.

Cheers,
Petr

[IV/JG7_4Shades]

Hi Petr,

Nice to hear what you are working on. Here are some comments.

First, in your template you may have used some Chiefs (mobile ground objects) to help define starting locations of some kinds of units. I think you said 420 of them! When initializing your campaign, that is no problem, SEOW will handle it fine. You will end up with 420 units "In Progress". However, when you try to view the battlefield in the MP, the MP will have to load and display all the units plus 420 sets of movement waypoints. This will take a long time, as you say. The remedy for this is to use the DCS "Remove In Progress" button. This will cancel ALL the movement orders, leaving all units at their respective starting locations, and the MP screen will load much faster. Essentially, the purpose of the template is to PLACE units in position on the map. Thereafter, human commanders issue movement orders.

Of course, it IS possible and OK to DEFINE initial movement orders via the template, and some scenarios demand this, but you have to be aware of the useability/playability issues this may cause, as you have found out.

1/ Maybe 4 types of trucks, 4 types of flak guns, 4 types of tanks, a few arty, 8 kinds of planes, 6 kinds of ships - that is a modest complexity of order of battle. IL-2 has a maximum memory demand of about 1 GB I think. If you hit the IL-2 memory limit the game will freeze all players.

2/ In the Object_Specifications table there is a FPS-friendly engagement range setting "FPS_Range" developed by Brandle. This might be a little too "short range", but you are free to edit the Engagement_Range and FPS_Range columns as you see fit. The DCS Environments tab allows you to select the kinds of ranges to use. Some designers limit use of heavy flak to industrial areas, keeping them away from combat actions in small towns and the countryside.

4/ Just give each aircraft type one or two loadouts, maximum. E.g. try to use SC bombs only for Axis, FAB only for Soviets, so the explosion effects need only be loaded once.

5/ I think aircraft parking of around 20% seems to give sparse but realistic looking airfields.

6/ Non-moving vehicles and tanks can either be stationary (not moving but shooting) or inactive (not moving and not shooting). By default, non-moving vehicles are stationary, but a certain percentage in each mission will be deemed to be "broken down" and will be converted to inactive for the duration of that mission. Next mission they can either be moved or remain non-moving, as normal. It is just a way to reduce the number of shooting objects in each mission, whilst keeping the number of valid targets the same.

8/ When you try to bomb a factory/fuel dump/railway station, SEOW will place a dummy target object there. If you destroy the dummy target, you will get a kill on the factory/fuel dump/railway station. There are two modes for the placement of dummy targets. The first is to place targets at ALL factory/fuel dump/railway station locations in EVERY mission. The second is only to place targets at those locations that have been explicitly target by bomber flight GATTACK waypoints in the MP. The first method generates one dummy target per location (maybe 450 in Normandy), the other method may only generate a few or none in each mission. Pros ^ Cons: first method means you can attack any installation at any time, but you have a larger and more complex mission file which may affect playability for some maps. The second method has a minimal set of objects, but only designated targets are LIVE and destroyable in any mission, so don't forget which targets are LIVE!

10/ In the template, place a single stationary tank. When you load the template with the DCS "Campaign Modes" tab "Load at Maximum Strength" option checked, it will appear in the MP as a single platoon of 4 tanks (Object_Specifications.Normal_Group_Strength). With the option unchecked, it will appear as a single platoon of 1 tank.

Cheers,
4S

[=gRiJ=Petr]

Hi 4Shades,

Thanks for your quick reply as always. Forgive my newbeeshness but off course I have more questions...

Hi Petr,

Nice to hear what you are working on. Here are some comments.

First, in your template you may have used some Chiefs (mobile ground objects) to help define starting locations of some kinds of units. I think you said 420 of them! When initializing your campaign, that is no problem, SEOW will handle it fine. You will end up with 420 units "In Progress". However, when you try to view the battlefield in the MP, the MP will have to load and display all the units plus 420 sets of movement waypoints. This will take a long time, as you say. The remedy for this is to use the DCS "Remove In Progress" button. This will cancel ALL the movement orders, leaving all units at their respective starting locations, and the MP screen will load much faster. Essentially, the purpose of the template is to PLACE units in position on the map. Thereafter, human commanders issue movement orders. After playtesting and finilizing the rules and OoB's I hope my squad will want to play it. After that I'll make the whole thing available if their is interest in the community.

Of course, it IS possible and OK to DEFINE initial movement orders via the template, and some scenarios demand this, but you have to be aware of the useability/playability issues this may cause, as you have found out.

Thanks. It's still very much a work in progress but the first draft of the OoB's and rules are finished and also the template with the initial deployments as stated before. We are now getting ready to start playtesting to verify the server performance, playability and victory objectives/scenario balancing. I have every intention of making this a massive campaign with 60-70 ground moves and 40 human pilots each side per turn. To this effect we have just rented a Quad core, 8 GB Ram, W2K3 R2 Standard server, 40Mb upload connection in a commercial datacenter for testing. It's about as heavy a machine I can afford. If there are SEOW server experts out there who could help us with getting the most out of this machine I could really use your help...

That is great info. So to recap, I can define chiefs for all mobile units but when importing remove them by using the DCS "Remove in progress" function, all the waypoints will be deleted after import and commanders can actually plot movement orders from turn one. Is this correct?

Just out of curiosity, the way around I was thinking of is to use static armour and/or define vehicles as artillery. This way no chief need to be plotted in the template. However, I'm not certain if I can import this in SEOW without negative effects. What do you think?

Also, will SEOW top up the armour to 4 per unit if I use stationary armour? This is important since I cannot select units of 4 when using stationary armour.

I'm sorry to push, but I really would like your opinion on whether there is any substantial improvement in the performance of the DB by using full SQL server 2005?

2/ In the Object_Specifications table there is a FPS-friendly engagement range setting "FPS_Range" developed by Brandle. This might be a little too "short range", but you are free to edit the Engagement_Range and FPS_Range columns as you see fit. The DCS Environments tab allows you to select the kinds of ranges to use. Some designers limit use of heavy flak to industrial areas, keeping them away from combat actions in small towns and the countryside.

Rgr, thanks!

4/ Just give each aircraft type one or two loadouts, maximum. E.g. try to use SC bombs only for Axis, FAB only for Soviets, so the explosion effects need only be loaded once.

Will do.

5/ I think aircraft parking of around 20% seems to give sparse but realistic looking airfields.

Will try!

6/ Non-moving vehicles and tanks can either be stationary (not moving but shooting) or inactive (not moving and not shooting). By default, non-moving vehicles are stationary, but a certain percentage in each mission will be deemed to be "broken down" and will be converted to inactive for the duration of that mission. Next mission they can either be moved or remain non-moving, as normal. It is just a way to reduce the number of shooting objects in each mission, whilst keeping the number of valid targets the same.

aha, gotcha!

8/ When you try to bomb a factory/fuel dump/railway station, SEOW will place a dummy target object there. If you destroy the dummy target, you will get a kill on the factory/fuel dump/railway station. There are two modes for the placement of dummy targets. The first is to place targets at ALL factory/fuel dump/railway station locations in EVERY mission. The second is only to place targets at those locations that have been explicitly target by bomber flight GATTACK waypoints in the MP. The first method generates one dummy target per location (maybe 450 in Normandy), the other method may only generate a few or none in each mission. Pros ^ Cons: first method means you can attack any installation at any time, but you have a larger and more complex mission file which may affect playability for some maps. The second method has a minimal set of objects, but only designated targets are LIVE and destroyable in any mission, so don't forget which targets are LIVE!

Ah, I understand now. I know I'm asking difficult questions but if you would have to put a percentage of extra load on the server if you place a target at all location on each mission versus the GATTACK option, how much would you say this is? 5%? 10%? ...?

Concerning the map, I'd like to use the Normandy2 map for this scenario. Is this map stable? Any issues I need to take into account?

10/ In the template, place a single stationary tank. When you load the template with the DCS "Campaign Modes" tab "Load at Maximum Strength" option checked, it will appear in the MP as a single platoon of 4 tanks (Object_Specifications.Normal_Group_Strength). With the option unchecked, it will appear as a single platoon of 1 tank.

Rgr, so you advice to load the template as individual tanks and not per set of 4. Could you elaborate on the reason for this?

Cheers,
Petr

[IV/JG7_4Shades]

Hi Petr,

Your server sounds big enough, but please make contact with 22GCT_Gross. Gross has extensive experience with running large SEOW campaigns on dedicated hardware.

SQL Server
SEOW is developed against MySQL and MSAccess, only. It has never been tested with SQL Server. You are on your own there! The only thing I can offer is that ADODB and ODBC connections are used, so there is a chance that SQL Server may work with SEOW.

Remove "In Progress"
Yes you are correct. The fundamental role of the template is just to place units on the map, so they can be used from mission one.

Just out of curiosity, the way around I was thinking of is to use static armour and/or define vehicles as artillery. This way no chief need to be plotted in the template. However, I'm not certain if I can import this in SEOW without negative effects. What do you think?

I don't understand this at all. In SEOW, tanks are tanks, vehicles are vehicles, whether they are stationary or not. It is up to the commanders to determine how they behave in each mission, e.g. by issuing movement orders, or positioning them to cover arcs defensively etc.

Also, will SEOW top up the armour to 4 per unit if I use stationary armour?

If you use the Load at Maximum Strength option at template initialization, then all tank units will be expanded to strength 4 (or whatever you define the relevant Normal_Group_Strength value to be) whether they are mobile or stationary. It is up to you whether you use this option or not. If you have factored the total number of objects into your campaign design already, then using the option reduces the effort involved in building your template by a factor of 4 (roughly)!

Normandy
This map, whilst strategically important, is a known poor performer in IL-2. It suffers from a memory leak, so it can be tough on PCs in complex battlefield scenarios. However, people have used it successfully in the past. Really, it comes down to play-testing your campaign before unleashing it for real. By comparison, Smolensk is beautifully well behaved. For factories, you will see the number of factories defined for Normandy in the Industrial_Installations table, 400+. That is a sizeable chunk of stationary target objects to carry around in each mission, but having them there all the time makes the tactical attack planning much easier!

Cheers,
4S

[22GCT_Gross]

Hello Petr,
glad of your new great server.
On my experience there are many factors to be considered to get a smooth playing:
1) on server side you are sitting on the best machine; you could assign the IL2 process at one core, keep graphics at the lowest settings and use the il2fb modified to use up to 2GB Ram (the standard one is set around 100 Mb)
2) on the bandwith side, you have no problem at all; we never check a traffic larger than 1.5 Mbit/sec
3) problems are always set on local machine; as I learned just from 69.GIAP, each player have to set connection at the same value. Furthermore, ou may need the il2fb modified too on local machine.

We always experienced memory limits kind of problems, or sometimes misworking due to high temperatures or graphic card low performance related to the graphical settings.

Anyway you need to limit the amount of data IL2 sever have to process and send to all players. Static objects are almost irrilevant, chiefs and ships are very relevant, map is absolutely relevant (especially if you're using MODS that require much more memory occupation)

It's useful to start a performance log service during a missoin, both on server and local machine, tracing bandwith, RAM and processor usage.

The best is always to test the server when it's full. I could send you some our successfully played missions, but you need a lot of player to test them

[=gRiJ=Petr]

Hello Gross,

Nice talking to you again and thanks very much for your reply, it is very useful information.

Would you perhaps be available for a TS conversation at the giap? Perhaps this evening? Besides the server, I would be very interested to talk to you about the upcoming multi sector campaign and the Normandy campaign I am designing now. The Italian squads are off course more than invited to participate in this campaign.

Do you know of a good and easy tool to use for the performance log or do you use windows perfmon?

Actually, I have managed to make a template which crashes my old and new server
It would be interesting if you could test it on your hardware to have a comparrison.

Hopefully speak too you soon.
Cheers,
Petr

[=gRiJ=Petr]

Hi 4Shades,

Thanks very much for all the great info and help!

If I ever get this campaign tested and play balanced, I will send you the template if you're interested.

Cheers
Petr

[=gRiJ=Petr]

Hi 4Shades,

Thanks very much for all the great info and help!

If I ever get this campaign tested and play balanced, I will send you the template if you're interested.

Cheers
Petr

[=gRiJ=Petr]

Hello Petr,
glad of your new great server.
On my experience there are many factors to be considered to get a smooth playing:
1) on server side you are sitting on the best machine; you could assign the IL2 process at one core, keep graphics at the lowest settings and use the il2fb modified to use up to 2GB Ram (the standard one is set around 100 Mb)
2) on the bandwith side, you have no problem at all; we never check a traffic larger than 1.5 Mbit/sec
3) problems are always set on local machine; as I learned just from 69.GIAP, each player have to set connection at the same value. Furthermore, ou may need the il2fb modified too on local machine.

We always experienced memory limits kind of problems, or sometimes misworking due to high temperatures or graphic card low performance related to the graphical settings.

Anyway you need to limit the amount of data IL2 sever have to process and send to all players. Static objects are almost irrilevant, chiefs and ships are very relevant, map is absolutely relevant (especially if you're using MODS that require much more memory occupation)

It's useful to start a performance log service during a missoin, both on server and local machine, tracing bandwith, RAM and processor usage.

The best is always to test the server when it's full. I could send you some our successfully played missions, but you need a lot of player to test them

Tushka pointed me to an old post explaining the memory leak problem which seems to affect the Normandy2 map as well. Has there been any imporvement on this front?
Could you explain a bit more about the modified exe which helps solve the problem? Where can I get it? Is it true that it is incorporated in the latest HSFX mod pack?
Thanks!

Cheers,
Petr

[22GCT_Gross]

about performance take a look at this old post:
http://www.22gct.it/forum/viewtopic.php?t=483
(as suggested by Rnzoli)

About memory leak there were many discussions about.
This was the first one: http://seowhq.net/seowforum/viewtopic.p ... light=#478
but others post can be found into this official forum.

You might send me what you are not able to run: database and template, or directly the mis file.

[=gRiJ=Petr]

Hi Gross, that was exactly the post Tushka game me and alerted me to the problem.

In you previous post you mention to use this modified exe. I'm more than willing to do that but where can I find it? Is it included in the recent HSFX package?

Thanks for offereing to run the mssion, let me first work out the DB problem, fine tune the new server and see what happens. I definitely would like to talk with you online regarding this so if you have any time available in the coming days pls send me an email at glennvc (at)telenet.be
Thanks!

Cheers!
Petr