Frequently Asked Questions about VMEX

If you do not know how to run VMEX, see the instructions.

FAQs

Q. It didn't work! What's wrong?

A. Dunno. First of all, make sure you are using the latest version of VMEX, since there are bugs in previous versions. Then read on.

Note that, if you're running in GUI mode, some errors only appear on the command line, and VMEX might seem to lock-up or do nothing when it hits an error. To check for this, run VMEX by clicking on "vmex.bat", and look at the command-line window (black window) for error messages while the program runs.

Q. I get an error saying "'java' is not recognized as an internal or external command"

A. You probably don't have the Java Runtime Environment installed properly. VMEX requires it to run.

Q. I get an error saying "Exception in thread "main" java.lang.UnsupportedClassVersionError" and then more stuff.

A. You have the wrong version of the Java Runtime Environment installed. Try entering "java -version" at the command prompt. If you don't see "build 1.5.0" (or higher) in there somewhere, you have the wrong version installed. Download JRE 5.0 and try installing again.

Q. I clicked on VMEX, and a black box popped-up quickly and disappeared. Why doesn't it work?

A. VMEX is a command line program, and can only be run from the command line. See the instructions for how to run it.

NEW! VMEX now includes a GUI if it is run without specifying any command-line options. Download newer versions from the main page.

Q. I can't get VMEX to work. Can you decompile this map for me and email me the .vmf file?

A. No. I will not distribute decompiled maps.

Q. I can't find any of the Valve maps to decompile. Where are they?

A. Probably stored in the .gcf file. You need GCF Scape to extract them, before they can be decompiled.

Q. Why do I get the message "Unknown map file ident or version"?

A. VMEX will only decompile .bsp files of maps made for the Half-Life 2 Source engine (e.g., for HL2, HL2DM, and Counter-Strike: Source). It cannot decompile maps made for Half-Life 1 (or its mods, such as Counter-Strike 1.6 and Condition Zero) or for other games which use the BSP format such as the Quake series.

The game Vampire: The Masquerade Bloodlines is based on the Source engine, but uses an earlier version of the bsp file format (17, rather than 19 as is used in HL2). As of v0.95, the decompiler partially supports this format. Some things, such as VTMB's modified entity I/O connections, are not handled correctly by the Hammer editor in the HL2 SDK. Note also that it is not possible to recompile these maps in a form suitable for VTMB unless somebody writes compiler tools that produce version 17 BSP files.

As of version 0.97, VMEX also has initial support for version 20 bsp files, as used for the HDR-enabled maps in the Day of Defeat: Source release.

Q. I've managed to load a map in Hammer, but I can't see anything in the 3D view (though I can see stuff in the 2D views). What's wrong?

A. The camera position defaults to the origin of the map (0,0,0). If there's no map brushes close enough to that position (within the 3D view's clipping range), the camera can't see anything. The easiest way to fix this is to select something in one of the 2D views, and use the "View / Center 3D view on selection" menu item.

Q. Why do I see a purple-and-black checkerboard texture on some/many brushes?

A. The map uses materials that Hammer can't find installed, or the materials are installed but the "Tool Texture" (the texture used inside Hammer to display that material) cannot be found. (The CS:S map de_aztec particularly suffers from this problem).

Also, maps that use custom textures/materials embedding into the .bsp file will need to have those files extracted from the BSP before Hammer can see them. You can do this with BSPZIP or Pakrat.

Q. What's this pink "invisible" texture I see everywhere?

A. If you are using the face-decompiling modes (-m0, -m1 or -m2) VMEX turns every visible surface in the map into a thin, pyramid-shaped brush (it's only 1 map unit thick by default, so it's difficult to tell that it's a pyramid). The "bottom" face of the pyramid is the original face in the map. The "side" faces of the pyramid, which don't exist in the original map, are by default textured with the pink "toolsinvisible" texture. If you don't like it, set the -s0 option for white sides, -s1 for black sides, or -s3 to use the same texture as the bottom face.

If you're using "brushes and planes" decompiling mode (-m3), VMEX recovers the whole brush for most brushes, so it doesn't need to use pyramids. The exception is displacement surfaces, where it must reconstruct the non-displacement sides from scratch, using pyramid brushes and the invisible texture.

Q. Hammer's message window complains about "solid with (number) sides was illegally in a visgroup". Is that bad?

A. This means means that a clip brush, which VMEX automatically assigns to the "Clip Brushes" visgroup, was actually part of an entity (and thus belongs in the "Brush Entities" visgroup). The error isn't important, since Hammer fixes it for you automatically.

Q. How do I set a command-line option?

A. Put it just before the name of the map. For example, type vmex -p mapname.bsp to decompile a map without any prop_static entities. You can see a list of available options here. Note that the default options work fine on almost all maps. If you're using the GUI, most of the options can be set from it without using the command-line switches.

Q. I have decompiled a map. When I try recompiling it again, all these errors occur! What's wrong?

A. Nothing's wrong. It's nearly impossible to for a decompiled map to exactly reproduce the original .vmf file, except for the very simplest of maps. VMEX is only designed to give you a way to look at the map's entities, etc., and not to produce recompilable maps. You will very likely not be able to recompile non-trivial maps without rebuilding many brushes.

Using Hammer's "check for problems" feature (Alt+P) will detect most of the brushes that have been malformed by decompiling. Once found, using the "fix" button will correct some errors, but often you must remake the brush.

There's one specific problem that will prevent re-compilation of a compiled map: if the original used func_areaportals, func_areaportalwindows, or func_occluders, they will not be tied to their brushes correctly and VBSP will fail. You will have to rebuild or delete any of these entities before you can compile the map.

Q. I was working on a map but accidently deleted my .vmf file. Can VMEX get it back for me from the .bsp file?

A. Perhaps. As stated above, VMEX recovers only an approximation of the original .vmf file (though using the default mode -m3 is much better than the face-generating modes 0,1 and 2). However, nearly all of your non-brush entities will be present, and you might be able to use the recovered walls as templates for rebuilding, but you will probably need to rebuild many brushes in the map. Also, of course, if the map is protected from decompiling (see below), VMEX will refuse to decompile it.

Q. Can I use VMEX to add more spawnpoints to a map?

A. You might be able to (bearing in mind the problems of recompiling the map as described above). However, I'd recommend using Entspy instead, which can add to & edit the entities of a compiled map, without the need to recompile it.

Q. I want add a kewl sniper tower to de_dust! Can I use VMEX to do that?

A. Don't do this. It will be difficult to do it, for the reasons given in the answers above. More importantly, it's very very bad form to recompile an existing map to "improve" it, even if you manage to do so. Also, it's most likely a copyright violation to distribute a map like this (yes, even if you don't sell it, and yes, even if you give the original author credit. To distribute any "derived work", you need explicit permission of the original's copyright holder - that would be Valve, for the official maps.)

Q. Talking of copyright, is it legal to do this?

A. I am not a lawyer, so cannot give legal advice. My understanding is that it's fine to decompile a map to see how it works, uses a particular entity, etc., but not to distribute the decompiled file or anything directly derived from it (such as sections cut-and-pasted into other maps). However, consult your own lawyer to be sure.

See the bottom of this forum thread for Valve's take on it. They seem to be fine with the program as long as you don't violate the EULA.

Q. Hammer tells me "(number) solids were not loaded" and/or I see missing walls, etc., in the decompiled map. Can I do anything to stop that?

A. Almost inevitably, not all brushes in a map can be recovered. You could try using one of the other decompiling modes (-m option). This seems to happen for some particularly complex brushes (e.g. ones that were made using the arch tool, or by carving) that are somewhat buggy.

Q. I see a blue "skip" texture on a lot of brushes. What is that for?

A. Often, it's used on the non-hint sides of a hint brush. Some maps (especially ones with big open areas, e.g. d2_prison_01.bsp) have lots of hint brushes, so try turning them off (uncheck visgroup "Hint Brushes") to see the map more clearly.

Q. VMEX stops with the message "This map is protected from decompiling." What does that mean?

A. Starting at version 0.80, VMEX will refuse to decompile maps that have certain special options set by their author. You can use an older version of VMEX to decompile it (which will do a much worse job of brush generation), or try to contact the map author if you really need to know how something was done.

Note, if you're running in GUI mode, you won't see this message unless you look in the command-line window.

Q. VMEX says this map is protected from decompiling. Can you decompile it and send it to me?

A. No. Sheesh.

Q. I'm a map author. How do I stop someone decompiling my map with VMEX?

A. There are several ways (in order of increasing security):

  1. Add a property key "no_decomp" with value "1" to any entity. (You will need to turn off smart edit to do this).
  2. Use a texture called "tools/locked" on any non-culled face in the map. (This is not a real texture, so you will have to make a custom one.) You may hide the texture in a sealed room, but it can't be on any culled face on the map (facing towards the void).
  3. Use this small prefab: protector.zip. Add it anywhere in the map, but don't rotate or scale it. If you retexture it, make sure all faces and all brushes have the same texture. Don't use the texture "tools/toolsinvisible", but "tools/toolsnodraw" is fine.

If VMEX detects any or all of these present in a map, it will refuse to decompile it. Note that a sufficiently wiley user can fairly easily alter your .bsp file to remove the first two methods (and thus allow the map to be decompiled), but they would need extensive knowledge of the .bsp format to remove the third, so this is the most secure.

Q. What are VMEX's limitations?

A.

  1. It can't (as of v0.98) tie areaportal brushes to their corresponding func_areaportal or func_areaportalwindow entities. This isn't too high priority, since you can tell usually tell what is an areaportal from it's texture, but it should be fixable.
  2. It can't recover any editor-specific information such as visgroups. Hammer's default visgroups work for some things, and VMEX places certain brushes (clip and hint brushes) in a visgroup so they can be turned off easily.
  3. It can't precisely recover to size and position of brushes. Rounding errors during the compiling/decompiling process may lead to tiny differences in position, which cause leaks and other problems if trying to recompile a decompiled map.
  4. All detail brushes are recreated individually (rather than many brushes being tied to one func_detail, as is more realistic). Unfortunately there is no grouping information retained in the BSP, so this is the best VMEX can do.
  5. Currently, info_overlay entities are only tied to their correct brush faces when decompiling in brush mode (-m3). Also, occasionally a decompiled overlay may be incorrectly oriented (flipped or mirrored).
Q. VMEX stops with the message: "java.lang.Exception in thread 'main'" (or something similar), and I'm sure that I have the correct version of the Java Runtime Environment installed.

A. You may have found a bug in VMEX. Please cut-and-paste the last few of lines of output, and email them to me along with the name of the map you were decompiling (and a place I can get the map, if it's not one of the official ones). My email address is printed by VMEX when you run it without specifying a map to decompile. Please don't send me the actual map, just tell where I can get it from.

Q. I know <Game X> is based on (or is a mod of) the Source Engine, so why can't VMEX decompile its maps?

A. Some Source Engine based games (and mods) modify the map format to be incompatible with standard HL2 maps. VMEX can't understand the modified structures of these maps, so can't decompile them correctly. Let me know and I may be able to release a version of VMEX which supports the changes, as I did for The Ship, for example.

Q. Why can't I decompile maps from HL2:Episode 2 or from TF2?

A. HL2:Ep2 and TF2 (and one map from Portal) use a new prop_static lump format. You need VMEX version 0.98g or later to decompile them correctly.


Back to the VMEX main page


Rof - (rof (at) bagthorpe.org)