Adventures in Virtual Reality Development

Thu Sep 25, 2014

This is a work in progress. Currently I am collecting references for a literature review.

Annotated Bibliography

van't Klooster, J.-W., van Beijnum, B.-J., Eliens, A., & Hermens, H. (2012). Interactive scenario visualization for user-based service development. Paper presented at the Collaboration Technologies and Systems (CTS), 2012 International Conference on.

The thesis of this paper is that specification and written prototype might be augmented by a visual representation of a project. This paper demonstrated that a 3D visual representation of a scenario, a medical software application aided in the development process, as evaluated by a panel of experts.

Continue Reading →

Söderman, M. (2005). Virtual reality in product evaluations with potential customers: An exploratory study comparing virtual reality with conventional product representations. Journal of engineering design, 16(3), 311-328.

This work deals with the virtual reality prototyping of an automotive model. It covers non-obvious drawbacks to this approach such as the effect of VR-naivete, or VR-novelty in evaluating products, where the resulting evaluation is more of the interface than of the product. Methodologically, the paper used semantic content analysis of user reviews of the prototype to assess the value of the input.

Pérez, F., & Valderas, P. (2009). Allowing end-users to actively participate within the elicitation of pervasive system requirements through immediate visualization. Paper presented at the Requirements Engineering Visualization (REV), 2009 Fourth International Workshop on.

This paper discusses a real-time system for visualizing requirements during the elicitation phase, emphasizing the importance of user interaction and feedback during this phase of development. A prototype tool was developed that allows visual interaction between the user and the engineer in developing requirements.

Escalona, M. J., & Koch, N. (2004). Requirements engineering for web applications-a comparative study. J. Web Eng., 2(3), 193-212.

This paper covers standard techniques for requirements engineering in the context of a web application, such as: interviewing, storyboarding, use case modeling, and concept mapping.

Gabrysiak, G., Giese, H., & Seibel, A. (2009, 1-1 Sept. 2009). Interactive Visualization for Elicitation and Validationn of Requirements with Scenario-Based Prototyping. Paper presented at the Requirements Engineering Visualization (REV), 2009 Fourth International Workshop on.

This is a straightforward position paper on the value of requirements prototypes in the elicitation and validation process. It is based on the thesis that 1) Users are extremely valid sources of requirements, 2) formal requirements models are insufficient tools for communicating with users. 3) Visualizations and prototypes combined with formal models are useful tools. It introduces the role of "requirements animator", IE maker of prototypes, or visual developer. The pathways discussed for the contribution of visual prototypes are: Validation of the total system specification (IE acceptance of prototype), tracing of used execution paths, versus non-useful ones, resolution of conflicting requirements...etc.

Warfel, T. Z. (2009). Prototyping: a practitioner's guide: Rosenfeld media.

annotation tba

Stone, R. J., Panfilov, P. B., & Shukshunov, V. E. (2011, 9-11 June 2011). Evolution of aerospace simulation: From immersive Virtual Reality to serious games. Paper presented at the Recent Advances in Space Technologies (RAST), 2011 5th International Conference on.

A review of serious game (SG) and Virtual reality (VR) projects in aerospace, drawing some interesting conclusions about virtual simulation: 1) The capacity to quickly prototype environments is valuable, built on pre-existing models. 2) There is an enhanced capacity for retention of information in VR. 3) cost effective 4) model quality is paramount for evaluation of model, not rendering system. (see Soderman paper)

Lowe, D., & Eklund, J. (2002). Client needs and the design process in web projects. J. Web Eng., 1(1), 23-36.

The thesis of this paper is that clients will develop their software requirements as they see designs, not a priori. Rather than answering the question "does this design meet the needs of...", we present designs in the context of the technological possibilities, and expanding and clarifying the goals of the project. This is a process centered on helping the client understand what they need, and then articulating it in a design. XP does this woven into the build, but this article suggests early stage prototyping as a way of evolving the requirements pre-build.

Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence-Teleoperators and Virtual Environments, 7(3), 225-240. doi: 10.1162/105474698565686

"Presence is defined as the subjective experience of being in one place or environment, even when one is physically situated in another." This paper theorizes about the factors contributing to presence, and lays out an empirical measure for evaluating a users experience therein.

Chung, L., Nixon, B., Yu, E., & Mylopoulos, J. (2000). Non-functional requirements. Software Engineering.

Mathematically f(x) = y is a transformation that says nothing about the time it takes to perform the operation, or other qualitative aspects to the process. (user experience, aesthetic quality, level of frustration, reliability, portability..etc) NFR covers these requirements.

Farhat, S., Simco, G., & Mitropoulos, F. J. (2009). Refining and reasoning about nonfunctional requirements. Paper presented at the Proceedings of the 47th Annual Southeast Regional Conference.

Classifies NFR's as Functionally Restrictive( FR-NFR: example, performance WRT memory usage), Additive Restrictive (AF-NFR: example, encryption), Policy Restrictive (PF-NFR: example, survivability), Architecture Restrictive (ArR-NFR: example, address system operation in a limited environment). note: do these come from Chung's original paper?

In Queue

Anderson, R. J. (1994). Representations and requirements: The value of ethnography in system design. Human-computer interaction, 9(3), 151-182.

Kaindl, H., Kramer, S., & Kacsich, R. (1998, 6-10 1998). A case study of decomposing functional requirements using scenarios. Paper presented at the Requirements Engineering, 1998. Proceedings. 1998 Third International Conference on.

Dourish, P. (2006). Implications for design. Paper presented at the Proceedings of the SIGCHI conference on Human Factors in computing systems.

Guo, J., Wang, Y., Zhang, Z., Nummenmaa, J., & Niu, N. (2012). Model-driven approach to developing domain functional requirements in software product lines. Software, IET, 6(4), 391-401. doi: 10.1049/iet-sen.2010.0072

Hughes, J., O'Brien, J., Rodden, T., Rouncefield, M., & Sommerville, I. (1995). Presenting ethnography in the requirements process. Paper presented at the Requirements Engineering, 1995., Proceedings of the Second IEEE International Symposium on.

Viller, S., & Sommerville, I. (1999). Social analysis in the requirements engineering process: from ethnography to method. Paper presented at the Requirements Engineering, 1999. Proceedings. IEEE International Symposium on.

Yi, J. S., ah Kang, Y., Stasko, J. T., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. Visualization and Computer Graphics, IEEE Transactions on, 13(6), 1224-1231.

Gabbard, J. L., & Swan, J. E. (2008). Usability Engineering for Augmented Reality: Employing User-Based Studies to Inform Design. Visualization and Computer Graphics, IEEE Transactions on, 14(3), 513-525. doi: 10.1109/TVCG.2008.24

Thu Sep 18, 2014

This is a work in progress, currently I am creating proof of concept experiments to make sure that the approach is viable, and collecting references for a literature review.

Scene from Star Wars, director: George Lucas

An idea for a immersive 3D representation of time sequenced geospatial data has emerged over the last couple of weeks. The thesis is:

A VR system for viewing timestamped geospatial data might have advantages over traditional 2D and 3D renderings in terms of the users comprehension of, and engagement with the data. This may be due to a few factors:

  • Novelty of the immersive representation as it effects the likelihood that the user will focus attention on the data at all.
  • Natural, non-abstracted user interaction, IE the use of tracked head movements or ambulation rather than traditional desktop mouse and keyboard based interactions, as they relate to reducing the cognitive burden of viewing the data.
  • Free viewing perspective, or the capacity to move the viewing platform in the scene, as it relates to traditional problems in 3D visualization like occlusion
  • Human scale reference cues as they effect quantitative perception. For example 3 blocks on a table, in a room, viewed with human stereoscopy are perceptually 3 times as high as 1 block. Scale perception becomes more difficult in abstraction (the pie chart is the oft maligned quintessence of this) with relationships in data becoming difficult to perceive.
  • Continue Reading →

    Methodologically, a rendering using Unity Game Engine that is derived from a database of both shape boundaries and data would be viewed on an Oculus Rift. An example database schema that matches experience with 2D products like GeoServer and ArcGIS is below.

    Example database schema

    Shape(int:id, varchar:name, varchar:vertexArray)
    Data(int:shape_id, datetime:timestamp, float:value)
    shape_id references

    This schema would allow for the viewing of a dataset, say obesity rates in the US, at different time frames. In a 'complete' dataset there would be a row in data with a common time stamp for every row in shape, perhaps multiplied by many timestamps at points in time. So for the obesity data, each state (if that is the level of resolution, county or census block could also be represented) would have a boundary defined in the Shape relation. There would be frames of data with a common timestamp in Data, say one for every year that there is data. Queries for operations involving dates (Aggregation, sum, difference...etc) could be built into the application, but probably won't make it into this version of the interface.

    Various modes for user interaction have been suggested from a flight simulator style interface to a model with human scale references, chairs around a table on which a representation of the data is sitting. This will be a subject of experimentation.

    Below is a log of the prototype development and evolution of the idea in the same style as development logs for the Bicycle-VR prototype and Immersive Genetic Algorithm Experiment.

    Development Log

    Making Parametric Meshes in Unity (Sep 18th)

    The first question to answer is how to programmatically make meshes (composed of triangles).

    A test extrusion of a simple L-shaped polygon. Notice the back face culling and missing cap.


    Polygon Triangulation for the Cap of the 3D Extrusion (Sep 24th)


    Next: Multiple extrusions of different colors

    Future, Rendering State Boundaries

    Note: clearly some of these shapes are concave, but do they have holes?


    Future: Reading from a database or translation layer Using Unity

    Resources to be included in the literature review

    Aigner, W., Miksch, S., Müller, W., Schumann, H., & Tominski, C. (2007). Visualizing time-oriented data—a systematic view. Computers & Graphics, 31(3), 401-409.

    Luckey, P. (2014). Porting Games to Virtual Reality. Steam Dev Days.

    Swan, J. E., Gabbard, J. L., Hix, D., Schulman, R. S., & Keun Pyo, K. (2003, 22-26 March 2003). A comparative study of user performance in a map-based virtual environment. Paper presented at the Virtual Reality, 2003. Proceedings. IEEE.

    Tominski, C., Schulze-Wollgast, P., & Schumann, H. (2005). 3d information visualization for time dependent data on maps. Paper presented at the Information Visualisation, 2005. Proceedings. Ninth International Conference on.

    Westerman, S. J., Cribbin, T., & Wilson, R. (2001). Virtual information space navigation: evaluating the use of head tracking. Behaviour & Information Technology, 20(6), 419-426. doi: 10.1080/01449290110069383

    Weber, M., Alexa, M., & Müller, W. (2001). Visualizing time-series on spirals. Paper presented at the Information Visualization, IEEE Symposium on.

Sat Aug 02, 2014

Standing at the Mean

Experimentation enabled by advances in low-cost consumer virtual reality hardware and software.

A few months ago, after a long hacking session with a genetic algorithm (an algorithm that evolves a solution from “chromosomes” over time), Unity Game Engine (a 3D video game engine) and an Oculus Rift immersive display, I had what I think is a unique experience: Creating a data set with the GA, writing a renderer that transformed the data into geometry, hues and color values, and piping the output to a head mounted display, I was able to don the goggles and somewhat literally walk around and stand at the mean of the data set and look around. For me, this view into the data was a transformative personal experience, if not a scientifically valid approach to understanding data...

Read the full article over on the Brandeis GPS Blog.
Loading more content.