It’s a tad annoying to be interested in both 2D game development and VR (which involves 3D). I can more easily make 2D sprites than 3D models. Animation is probably easier to handle in 3D than 2D, since you don’t have to draw as much, but the modeling experience is a bit against me. Until then, I’ve installed myself Unity and played around with the demos.
More than that, I tried to apply an equirectangular projection camera to Unity via five separate fisheye cameras and splicing it together with a bit of code. Result is a believable equirectangular image of a 360 degree field of view around the camera point of view, without much in the way of artifacts (except when there’s special lighting effects involved, and even then, there’s only a bit of bad stuff at the seams then).
Example of an equirectangular 360 projection
Now, problem with having 5 separate cameras in a Unity game (or any game for that matter) is that you generally have to render those 5 different cameras at the same time. That, understandably, takes quite a lot of graphics juice out of your computer. It works just fine if you have a mighty rig and the scene isn’t that complex, but once you add some complexity, your FPS will plummet like a lemming off a cliff.
The second thing is how the cameras work – they will replace your main camera, and in order to maintain a good video, you have to be looking straight ahead, and never look up and down, since the video watcher will probably go dizzy if you do and the general point of view of the video will be tilted if the watcher looked to the side. You can see an example of that here:
The other matter is that some games just won’t let you have multiple cameras active at the same time. That might just be my own limited technical experience with Unity, but the cameras just show the backdrop and nothing else. I suppose it’s a matter that could use more research on my part.
Another thing is the matter of game perspective – an FPS game would work well enough with this kind of recording system, but you can’t look up or down to aim your guns (unless the aiming was a separate system, like in ARMA or any rail shooter). Racing games come to mind as being very good at being recorded in this way; you can see an example of that here:
Cockpit games are good fits for VR already, so they would work well enough in this recording system as well. I have applied this sort of camera to an isometric-style third person shooter, but you can’t look around (not that you need to with 360 recording) nor aim your gun without more modification in the code. The monsters seem to be glitching out due to it being an older version of Unity. The camera system I’m using is only compatible with 4.6 onwards. Here’s the video in question:
There is also the matter of resolution. Due to how the projection system on the video works, it’s best to have a 2:1 aspect ratio. Most games don’t support a 2:1 aspect ratio. You can set up custom resolutions, but it’s a pain in the butt. Even in Unity, where you make your own game, you still have to change some things in the registry editor in order to maintain the resolution you set for the final application (for some reason). Also, say you’re using a standard HD video resolution for equirectangular shooting: 1920×960. That looks and works well enough, but you might get quite a bit of pixellation as you look at the video, as the vertical field of view is quite a bit lower.
There are other projection systems that would look well for VR video, but I’ll go into those next time.