As you may know, you can get support for Direct3D in Windows guests running in VirtualBox through the use of Wine's Direct3D OpenGL compatibility layer, WineD3D.
The main problem is getting the installer to work. Even if you know, as I do, how to cross-compile on Linux, I still recommend downloading the pre-compiled binary installer because it helps you to know where all the files go on Windows.
Most seem to be recommending installation in Safe Mode, which only partially works; even Administrator doesn't have permission to overwrite some of the Windows binaries. Rather than beating yourself over the head running the Windows installer, do yourself a favor and download the ISO9660 image, and then, if you don't have one already, obtain a copy of SystemRescueCD, which, if you support Windows machines, you really, really need one of these. Trust me. In this case, you don't have burn it, just download the ISO and put it somewhere convenient.
Okay, here we go.
Launch VirtualBox and edit the settings in your VM. Add the System Rescue CD ISO as one of your CD devices. Add another CD device (if you don't already have two), and then map that one to the WineD3D ISO:
Next, you want to make sure that the boot order is set to boot the CD:
Finally, if you haven't already done so, you need to enable 3D acceleration support. Enabling 2D acceleration support is also a good idea. You need to to set aside at least 32 MB of RAM for video. I 'd go with 64 MB, if possible.
Once you've got all your settings, you need to boot the System Rescue CD. When you (finally) get to a shell prompt, mount your Windows C: drive with something like:
|mount /dev/sr1 /mnt/custom|
|mount /path/to/somewhere/writeable /mnt/gentoo # (optional) |
|cp '$SYSDIR' /mnt/windows/Windows/System32 |