If you do not know how to run VMEX, see the instructions.
A. Dunno. First of all, make sure you are using the latest version of VMEX, since there are bugs in previous versions. Then read on.
Note that, if you're running in GUI mode, some errors only appear on the command line, and VMEX might seem to lock-up or do nothing when it hits an error. To check for this, run VMEX by clicking on "vmex.bat", and look at the command-line window (black window) for error messages while the program runs.
A. You probably don't have the Java Runtime Environment installed properly. VMEX requires it to run.
A. You have the wrong version of the Java Runtime Environment installed. Try entering "java -version" at the command prompt. If you don't see "build 1.5.0" (or higher) in there somewhere, you have the wrong version installed. Download JRE 5.0 and try installing again.
A. VMEX is a command line program, and can only be run from the command line. See the instructions for how to run it.
NEW! VMEX now includes a GUI if it is run without specifying any command-line options. Download newer versions from the main page.
A. No. I will not distribute decompiled maps.
A. Probably stored in the .gcf file. You need GCF Scape to extract them, before they can be decompiled.
A. VMEX will only decompile .bsp files of maps made for the Half-Life 2 Source engine (e.g., for HL2, HL2DM, and Counter-Strike: Source). It cannot decompile maps made for Half-Life 1 (or its mods, such as Counter-Strike 1.6 and Condition Zero) or for other games which use the BSP format such as the Quake series.
The game Vampire: The Masquerade Bloodlines is based on the Source engine, but uses an earlier version of the bsp file format (17, rather than 19 as is used in HL2). As of v0.95, the decompiler partially supports this format. Some things, such as VTMB's modified entity I/O connections, are not handled correctly by the Hammer editor in the HL2 SDK. Note also that it is not possible to recompile these maps in a form suitable for VTMB unless somebody writes compiler tools that produce version 17 BSP files.
As of version 0.97, VMEX also has initial support for version 20 bsp files, as used for the HDR-enabled maps in the Day of Defeat: Source release.
A. The camera position defaults to the origin of the map (0,0,0). If there's no map brushes close enough to that position (within the 3D view's clipping range), the camera can't see anything. The easiest way to fix this is to select something in one of the 2D views, and use the "View / Center 3D view on selection" menu item.
A. The map uses materials that Hammer can't find installed, or the materials are installed but the "Tool Texture" (the texture used inside Hammer to display that material) cannot be found. (The CS:S map de_aztec particularly suffers from this problem).
Also, maps that use custom textures/materials embedding into the .bsp file will need to have those files extracted from the BSP before Hammer can see them. You can do this with BSPZIP or Pakrat.
A. If you are using the face-decompiling modes (-m0, -m1 or -m2) VMEX turns every visible surface in the map into a thin, pyramid-shaped brush (it's only 1 map unit thick by default, so it's difficult to tell that it's a pyramid). The "bottom" face of the pyramid is the original face in the map. The "side" faces of the pyramid, which don't exist in the original map, are by default textured with the pink "toolsinvisible" texture. If you don't like it, set the -s0 option for white sides, -s1 for black sides, or -s3 to use the same texture as the bottom face.
If you're using "brushes and planes" decompiling mode (-m3), VMEX recovers the whole brush for most brushes, so it doesn't need to use pyramids. The exception is displacement surfaces, where it must reconstruct the non-displacement sides from scratch, using pyramid brushes and the invisible texture.
A. This means means that a clip brush, which VMEX automatically assigns to the "Clip Brushes" visgroup, was actually part of an entity (and thus belongs in the "Brush Entities" visgroup). The error isn't important, since Hammer fixes it for you automatically.
A. Put it just before the name of the map. For example, type vmex -p mapname.bsp to decompile a map without any prop_static entities. You can see a list of available options here. Note that the default options work fine on almost all maps. If you're using the GUI, most of the options can be set from it without using the command-line switches.
A. Nothing's wrong. It's nearly impossible to for a decompiled map to exactly reproduce the original .vmf file, except for the very simplest of maps. VMEX is only designed to give you a way to look at the map's entities, etc., and not to produce recompilable maps. You will very likely not be able to recompile non-trivial maps without rebuilding many brushes.
Using Hammer's "check for problems" feature (Alt+P) will detect most of the brushes that have been malformed by decompiling. Once found, using the "fix" button will correct some errors, but often you must remake the brush.
There's one specific problem that will prevent re-compilation of a compiled map: if the original used func_areaportals, func_areaportalwindows, or func_occluders, they will not be tied to their brushes correctly and VBSP will fail. You will have to rebuild or delete any of these entities before you can compile the map.
A. Perhaps. As stated above, VMEX recovers only an approximation of the original .vmf file (though using the default mode -m3 is much better than the face-generating modes 0,1 and 2). However, nearly all of your non-brush entities will be present, and you might be able to use the recovered walls as templates for rebuilding, but you will probably need to rebuild many brushes in the map. Also, of course, if the map is protected from decompiling (see below), VMEX will refuse to decompile it.
A. You might be able to (bearing in mind the problems of recompiling the map as described above). However, I'd recommend using Entspy instead, which can add to & edit the entities of a compiled map, without the need to recompile it.
A. Don't do this. It will be difficult to do it, for the reasons given in the answers above. More importantly, it's very very bad form to recompile an existing map to "improve" it, even if you manage to do so. Also, it's most likely a copyright violation to distribute a map like this (yes, even if you don't sell it, and yes, even if you give the original author credit. To distribute any "derived work", you need explicit permission of the original's copyright holder - that would be Valve, for the official maps.)
A. I am not a lawyer, so cannot give legal advice. My understanding is that it's fine to decompile a map to see how it works, uses a particular entity, etc., but not to distribute the decompiled file or anything directly derived from it (such as sections cut-and-pasted into other maps). However, consult your own lawyer to be sure.
See the bottom of this forum thread for Valve's take on it. They seem to be fine with the program as long as you don't violate the EULA.
A. Almost inevitably, not all brushes in a map can be recovered. You could try using one of the other decompiling modes (-m option). This seems to happen for some particularly complex brushes (e.g. ones that were made using the arch tool, or by carving) that are somewhat buggy.
A. Often, it's used on the non-hint sides of a hint brush. Some maps (especially ones with big open areas, e.g. d2_prison_01.bsp) have lots of hint brushes, so try turning them off (uncheck visgroup "Hint Brushes") to see the map more clearly.
A. Starting at version 0.80, VMEX will refuse to decompile maps that have certain special options set by their author. You can use an older version of VMEX to decompile it (which will do a much worse job of brush generation), or try to contact the map author if you really need to know how something was done.
Note, if you're running in GUI mode, you won't see this message unless you look in the command-line window.
A. No. Sheesh.
A. There are several ways (in order of increasing security):
If VMEX detects any or all of these present in a map, it will refuse to decompile it. Note that a sufficiently wiley user can fairly easily alter your .bsp file to remove the first two methods (and thus allow the map to be decompiled), but they would need extensive knowledge of the .bsp format to remove the third, so this is the most secure.
A.
A. You may have found a bug in VMEX. Please cut-and-paste the last few of lines of output, and email them to me along with the name of the map you were decompiling (and a place I can get the map, if it's not one of the official ones). My email address is printed by VMEX when you run it without specifying a map to decompile. Please don't send me the actual map, just tell where I can get it from.
A. Some Source Engine based games (and mods) modify the map format to be incompatible with standard HL2 maps. VMEX can't understand the modified structures of these maps, so can't decompile them correctly. Let me know and I may be able to release a version of VMEX which supports the changes, as I did for The Ship, for example.
A. HL2:Ep2 and TF2 (and one map from Portal) use a new prop_static lump format. You need VMEX version 0.98g or later to decompile them correctly.
Rof - (rof (at) bagthorpe.org)