Login x
User Name:
Password:
Social Links Facebook Twitter YouTube Steam RSS News Feeds

Members Online

»
0 Active | 77 Guests
Online:

LATEST FORUM THREADS

»
CoD: Battle Royale
CoD+UO Map + Mod Releases
Damaged .pk3's
CoD Mapping
heli to attack ai
CoD4 SP Mapping

Forums

»

Welcome to the MODSonline.com forums. Looking for Frequently Asked Questions? Check out our FAQs section or search it out using the SEARCH link below. If you are new here, you may want to check out our rules and this great user's guide to the forums and the website.
For more mapping and modding information, see our Wiki: MODSonWiki.com

Jump To:
Forum: All Forums : Call of Duty 2
Category: CoD2 Scripting
Scripting and coding with Call of Duty 2.
Moderators: foyleman, Foxhound, Mystic, StrYdeR, batistablr, Welshy, DrBiggzz, supersword, playername
Latest Posts
Subscribed Posts
Search
Restricted Access Topic is Locked subscribe
Author Topic: IWD Checksum Analysis by Bullet-Worm
Tally
General Member
Since: Apr 21, 2005
Posts: 819
Last: Oct 26, 2012
[view latest posts]
Level 7
Category: CoD2 Scripting
Posted: Tuesday, Aug. 28, 2012 08:36 am
Since the demise of IWNation.com, the original post made by Bullet_Worm about the dreaded check-sum error in COD2 is now lost in time. It was a pioneering post, as at that time, no one really knew what caused it or how we could work around it. Worm's analysis made it possible for us to run lots of mods and custom maps on our servers.

Given its importance, I therefore hereby reproduce it for posterities sake:

Dated: June 15th 2006:
--------------------------------------

Results of my experiments with IWD Sum Mismatch

This is going to be a LONG post. However, for server admins the information below is very important, ESPECIALLY if they are fighting with the now infamous IWD Sum/Mismatch error.

The ACTUAL error appears to be caused because the info_string is cut-off at 1024 characters, and you get the IWD Sum/Mismatch error because the client does not recieve all of the checksums it needs from the server to verify its files are identical. As a side note, the 'loop-loading' effect of the info_string appears to come into play when the entire list of checksums makes it in the info_string, but the info_string has still been cut-off AFTER the checksum portion of the string (as you can see below).

Let me start by showing you a COMPLETE info_string:

Code:

\bg_fallDamageMaxHeight\480\bg_fallDamageMinHeight\252\g_synchronousClients\0\sv_cheats\0\sv_disableClientConsole\0\sv_iwdNames\z__svr_csv z_svr_sound_aliases_for_dhc_map_pack_4 z_svr_sound_aliases_for_dhc_map_pack_2 z_svr_sound_aliases_for_dhc_map_pack_02 z_svr_sound_aliases_for_dhc_map_pack_01 z_svr_pam____202 z_dhc_mp1 zzz_mp_powcamp iw_15 iw_14 iw_13 iw_12 iw_11 iw_10 iw_09 iw_08 iw_07 iw_06 iw_05 iw_04 iw_03 iw_00\sv_iwds\-675245519 1058077270 1058077270 1058077270 1058077270 -635133705 2070676332 -149688453 181429573 -1449716526 780394069 -1333623355 -1980843666 1334775335 -621896007 1101180720 1046874969 1053665859 1842349204 -1652414412 1659111092 178615151 \sv_pure\1\sv_referencedIwdNames\main/z__svr_csv main/z_svr_sound_aliases_for_dhc_map_pack_4 main/z_svr_pam____202 main/iw_15 main/iw_13 main/iw_08 main/iw_07 main/iw_06 main/iw_03\sv_referencedIwds\-675245519 1058077270 -635133705 181429573 780394069 1101180720 1046874969 1053665859 1659111092 \sv_serverid\16\sv_voice\0\sv_voiceQuality\1\timescale\1


Note that this info string is 1026 characters long, just two characters over the 1024 limit. I have confirmed this is the entire info_string by removing two characters in a non-referenced IWD file name (importance to be discussed later) and verified that with those two characters removed, the server would NOT come up with the info_string error.

Using this knowledge, we now we know EXACTLY what is contained in an info_string. We can use that information to figure out what we server admins can do to REDUCE the length of this string.

I should note, these are the results based on a WINDOWS SERVER! I do not run a Linux server so your results may or may not match mine. Also, I PERSONALLY will not be "supporting" this technique. The information is here for you to discern and use on your own. Not all of my suggestions are for the un-experienced server admin and that is all I will say about that.

First, there is nothing we can do about a large majority of the settings contained in the string. These (and their respective settings) must be sent. Nothing we (the CoD2 user) can do about it.

Code:

bg_fallDamageMaxHeight
bg_fallDamageMinHeight
g_synchronousClients
sv_cheats
sv_disableClientConsole
sv_pure
sv_serverid
sv_voice
sv_voiceQuality
timescale


Luckily, those only add up to approximately 188 characters worth of our info_string, which leaves us with about 836 characters to play with.

With the rest of the info_string, we can play Damage Control. Or what I like to call in this case... "ghetto-riggin'"

We have FOUR settings left that are in the info_string that we can effect and I will look into each one of these seperately.


1) sv_iwdNames

Code:

sv_iwdNames\z__svr_csv z_svr_sound_aliases_for_dhc_map_pack_4 z_svr_sound_aliases_for_dhc_map_pack_2 z_svr_sound_aliases_for_dhc_map_pack_02 z_svr_sound_aliases_for_dhc_map_pack_01 z_svr_pam____202 z_dhc_mp1 zzz_mp_powcamp iw_15 iw_14 iw_13 iw_12 iw_11 iw_10 iw_09 iw_08 iw_07 iw_06 iw_05 iw_04 iw_03 iw_00\


This is a list of ALL IWD file names found on your server (or MY test server for the above example).

Note: "localized_" IWD filenames are not included in the info_string at all, so messing with these is useless as the server does not even recognize them for these purposes.

What can we do to reduce the number of characters here?

A) Remove UNNECESSARY IWD files.
If you look carefully in the list of IWD names above, you will see that iw_01 and iw_02 are missing. They aren't missing. I REMOVED them from the server. They contain single player maps and are useless for my dedicated multiplayer server. I have also found that a DEDICATED (Windows) server is not hindered by removing iw_11 and iw_12! For me, that makes 4 STOCK IWD file names I can remove from the server. :D

Also, make sure you are not harboring OLD versions of mods that are not being used, as this does nothing but add to this list of IWD names.

B) Use short names for IWD files.
IW actually helped us a little by keeping their stock IWD files down to 5 characters each, but don't give them too much credit as they ended up using 15 IWD names which REALLY kicks the info_string in the pants (as I am about to show).

C) Combine mods where you can.
Try to only have ONE server-side IWD file and ONE client-side IWD file if you are using mods.

D) For servers with access to fs_game folders only:
You can EXTRACT the serverside component of your mod to your server's fs_game folder and completely REMOVE the IWD name from the info_string. This is the 'old' method of creating "server-side-only" mods before the 1.2 patch gave us back the '_svr_' naming convention.

E) For servers without fs_game access, but access to high-speed HTTP Re-direct:
COMBINE your server-side and client-side components of your mod into ONE IWD file. Sure, the client will have to download the whole mod, but with HTTP redirect it should not take more than a second or so either way and you SAVE an IWD name from being in the info_string


2) sv_iwds

Code:

sv_iwds\-675245519 1058077270 1058077270 1058077270 1058077270 -635133705 2070676332 -149688453 181429573 -1449716526 780394069 -1333623355 -1980843666 1334775335 -621896007 1101180720 1046874969 1053665859 1842349204 -1652414412 1659111092 178615151 


These are the in-game checksums; The numbers IW uses to verify that the IWD files on the server are exactly the same as what the clients have. There is one entry for EACH sv_iwdnames listing. Here is where those 15 seperate IWD names IW left us come back to haunt us. They are 9-11 characters EACH (NOT including spacing!!!)! Thus for those 15 STOCK IWD files IW has given us, we just ate up about another 180 characters of our available info_string. OUCH!

What can we do to reduce the number of characters here?

A) AGAIN, remove UNNECESSARY IWD files. Anything you can do to REDUCE the number of IWD names, reduces the number of IWD checksums!
Not only do you save on the sv_iwdNames space, but you also eliminate 10-12 characters worth of checksum space for each file removed. Thus just by removing iw_01, iw_02, iw_11, and iw_12 I didn't just save 20 characters of info_space, but 64 characters of info_string space!

B) Combining mods into ONE IWD file where possible will obviously reduce the number of IWD names, but it also reduces the number of checksums! Double bonus for this!

C) For servers with access to fs_game folders only: Extracting serverside only sections of mods and removing an IWD file also gets rid of THAT checksum in the info_string. BONUS!

**Note about sv_iwdNames and sv_iwds as they apply to Custom Maps**
There is nothing we can do about this. The name of the custom map IWD will be sent no matter what, the place we can save info_space will be in the sv_referencedIWDnames as I will show in a minute.


3) sv_referencediwdNames

Code:

sv_referencedIwdNames\main/z__svr_csv main/z_svr_sound_aliases_for_dhc_map_pack_4 main/z_svr_pam____202 main/iw_15 main/iw_13 main/iw_08 main/iw_07 main/iw_06 main/iw_03


This is a list of all IWD filenames being actively 'loaded' by the server. Notice the word 'loaded' because it becomes very important when we apply it to a custom maps server.

What can we do to reduce the number of characters here?

A) AGAIN!!! Combining mods into ONE IWD file will reduce the number of IWD names.

B) Short file names are important too, as the number of characters in 'loaded' IWD files actually count against us in the info_string TWICE. Once in sv_iwdNames and once in sv_referencediwdNames.

C) For servers with access to fs_game folders only: You can EXTRACT the serverside component of your mod to your server's fs_game folder and completely REMOVE the IWD name from this part of the info_string.

D) For servers without fs_game access, but access to high-speed HTTP Re-direct, COMBINE your server-side and client-side components of your mod into ONE IWD file. Sure, the client will have to download the whole mod, but with HTTP redirect it should not take more than a second or so either way and you SAVE an IWD name from being in the info_string TWICE!

3) sv_referencedIwds

Code:

sv_referencedIwds\-675245519 1058077270 -635133705 181429573 780394069 1101180720 1046874969 1053665859 1659111092 


These are the checksums for the 'loaded' IWD files. It is a waste from IW since they are essentially REPEATS of the numbers available in sv_iwds, but I guess it made it a little easier to match the names up with the checksums and they probably did not anticipate the info_string length problem...

What can we do to reduce the number of characters here?
The same information that applies to sv_iwds applies here except for the removal of unneccessary IWD files (since they will not get 'loaded' anyway)

A) Combining mods into ONE IWD file where possible will obviously reduce the number of IWD names, but it also reduces the number of checksums! If the IWDs combined get actively 'loaded', QUADRUPLE BONUS! You just eliminated wasted space in sv_iwdnames, sv_iwds, sv_referencediwdNames, AND sv_referencediwds. Anything you can do to reduce the number of IWD names being actively 'loaded' will greatly reduce the amount of characters used in your info_string.

B) For servers with access to fs_game folders only: Extracting serverside only sections of mods and removing an IWD file also gets rid of THAT checksum in the info_string. BONUS!

NOW! For an important note about Custom Maps and what I will call the "sound csv bug":

For those of you unaware: There is a problem in CoD2 where the server automatically 'loads' (see the REFERENCED IWD files above) ALL custom map's soundaliases/*.csv files even if that custom map is not actively being played.

This causes TWO problems:
1) When a client connects, it has to download ALL of the 'loaded' (referenced) IWD files that the server has. This means that a client would have to download EVERY SINGLE CUSTOM MAP OR MAP PACK before the client can begin playing even if the server is currently running a STOCK map!

2) The second problem DIRECTLY relates to our IWD Sum/Mismatch problem. If you have EVERY custom map 'loaded' due to the sound csv bug, then EVERY custom map will be listed not only in your sv_iwdnames and sv_iwds, but ALSO in your sv_referencediwdNames AND sv_referencediwds. OUCH!

To work around the sound csv bug, which will allow your players to download ONLY the map you are actually running AND save you LOTS of info_string space, you can do one of the following:

Option 1: BEST Option but only available to those of you who have access to the use of an fs_game folder -
1) Make a directory in your fs_game folder named "soundaliases" (without the quotes!)
2) Open EACH custom map (or mappack) IWD file and extract the contents of the soundaliases directory and put those files in your fs_game soundaliases directory. DO NOT DELETE THE FILES FROM THE MAP IWD FILE.
3) Put all the un-altered custom map IWD files in your /main directory.

Option 2: Not the BEST option because it involves more steps and adds an IWD file, but if you don't have access to the fs_game folder and run custom maps it is still WAY better.
1) Make a new directory (folder) in your C:\ and name it "soundaliases" (without the quotes)
2) Open EACH custom map (or mappack) IWD file and extract the contents of the soundaliases directory and put those files in your newly created soundaliases directory. DO NOT DELETE THE FILES FROM THE MAP IWD FILE.
3) Once you have all the soundaliases/*.csv files accounted for, ZIP your newly created soundaliases directory. Make sure when you open the Zip file, there is a soundaliases directory in there and the csv files are in the soundaliases directory. It is very important that the directory structure for your new zip file be the same as it was in the original custom map files.
4) Rename your Zip file to "z__svr_" + some custom text specific to your server (please note that there are TWO underscores after the 'z', this is intentional). Try to be original with the custom text part of the filename so that you do not conflict with another server.

Note: If you really want to be chinchy with info_string space and do not care if clients have to download the file, you can live without the _svr_ part of the filename, but still leave two underscores AFTER the 'z'.

Note 2: It can also be COMBINED with a mod IWD file so as not to effect the length of the info_string, but keep in mind that every time you change the custom maps on your server you will have to re-combine all of your mods to re-create the ONE IWD file goal.
5) Remove the part of the Zip filename that says ".zip" and replace it with ".iwd"
6) Upload your newly created IWD file to your server's \main directory along with all of your custom maps.

------------------------------------------------------------

I hope this has been educational for those of you who understand what I am saying. If what I have said is going WAY over your head, you may not be ready to do anything with this information. Again, I DO NOT WANT to spend the next 3-4 months of my life trying to get everyone's server optimized for running the MAXIMUM number of custom maps. I will answer questions as I am able, and I am sure others in this community who understand my post will help out as they can as well.

---------------------------------------

If anyone wants to read the original as it was in-situ, you can read it here:

http://web.archive.org/web/20071030195408/http://www.iwnation.com/Forums/index.php?showtopic=24161
Share |
IzNoGoD
General Member
Since: Nov 29, 2008
Posts: 694
Last: Nov 10, 2012
[view latest posts]
Level 6
Category: CoD2 Scripting
Posted: Tuesday, Aug. 28, 2012 11:14 am
I just yesterday thought about the very same problem.

Your solution in 1.E) is a bit tricky though, as most people prefer not to have their serverside scripts exposed (available for download).

You can add the following to the possible solutions:
As iw_07.iwd is quite small (less than 3mb) you can easily include said file in your mod and remove the original (add the contents of iw_07 to your clientside part of the mod).

The iw_11 and iw_12 contain directx 9 images. Have you tried running a server and connecting with a dx9 enabled client?
Share |
Tally
General Member
Since: Apr 21, 2005
Posts: 819
Last: Oct 26, 2012
[view latest posts]
Level 7
Category: CoD2 Scripting
Posted: Tuesday, Aug. 28, 2012 12:05 pm
IzNoGoD writes...
Quote:
I just yesterday thought about the very same problem.

Your solution in 1.E) is a bit tricky though, as most people prefer not to have their serverside scripts exposed (available for download).

You can add the following to the possible solutions:
As iw_07.iwd is quite small (less than 3mb) you can easily include said file in your mod and remove the original (add the contents of iw_07 to your clientside part of the mod).

The iw_11 and iw_12 contain directx 9 images. Have you tried running a server and connecting with a dx9 enabled client?


Quote:
Your solution in 1.E) is a bit tricky though, as most people prefer not to have their serverside scripts exposed (available for download).


You don't need an IWD compressed file for server side files. Just keep them on a server in uncompressed folders. We've been doing that since vCOD, before the COD 1.5 and the introduction of _svr_ PAK3 files with COD:UO. The eXteme+ mod for COD2 still uses the uncompressed folder arrangement for setting up server-side only files. This is why it is never a good idea trying to mod in a vacuum, without looking at how other mods are doing things. Check out as many mods as possible. That is the best recipe for good modding education there is.

Quote:
The iw_11 and iw_12 contain directx 9 images. Have you tried running a server and connecting with a dx9 enabled client?


As soon as Worm had posted his analysis, people corrected him about several items - one of which was the need for DX9 images. In those days - 2006 - hardly anyone ran DX9 as our machines couldn't handle it. So, Worm can be forgiven for testing in DX7 only: most of us did.

BTW - please don't address your comments to me, as if I wrote that post. As is made abundantly clear, it is Bullet_Worm's post. Worm was a truly great modder in the COD community. Inventor of the PAM competitive mod, and inventor of Worm's "Power Server" mod - a great COD2 mod which pioneered many features we now take for granted in COD2 mods.
Share |
Samuel033
General Member
Since: Dec 10, 2009
Posts: 484
Last: Dec 25, 2015
[view latest posts]
Level 5
Category: CoD2 Scripting
Posted: Friday, Dec. 25, 2015 10:24 pm
Old post I know. But just found it believe it or not. Later on going to have to spend more time going through it to hopefully help with some of our checksum problems on some maps.

One interesting tho. In the dvar for all the iwds, they are listed twice in the sv_iwdNames string. They are also listed twice in the console log for the search path. Which I assume it taken from the dvar for which ones to load. But I do not see this elsewhere in servers in other locations with the same mod.

Any idea why?

edited on Dec. 25, 2015 03:27 pm by Samuel033

This is for cod4 but I assume the above would apply there as well. Or at least as a starting point.
Share |
Restricted Access Topic is Locked subscribe
MODSonline.com Forums : Call of Duty 2 : CoD2 Scripting

Latest Syndicated News

»
Codutility.com up and runn...
Nice, and there still using the logo and template for the screenshots, which...
Codutility.com up and runn...
dundy writes...Quote:Call of Duty modding and mapping is barly alive only a ...
Codutility.com up and runn...
Mystic writes...Quote:It seems to me the like the site is completely dead? ...
Codutility.com up and runn...
It seems to me the like the site is completely dead?

Partners & Friends

»