'Digital Embryos'

Digital embryos are naturalistic, virtual 3-D objects that have significant potential uses in the study of perception and perceptual learning (Hauffen et al., 2011).

Digital embryos are created by simulating some of the key processes of embryonic development, such as morphogen-mediated cell division, cell growth, cell movement and programmed cell death. A given set of simulation parameters (or 'genotype') creates a unique, novel and naturalistic 3-D shape (Brady and Kersten, 2003). Since this creates shapes de novo by simulating natural morphogenetic processes, we refer to this as the Virtual Morphogenesis (VM) process. Each unique genotype in VM creates a unique digital embryo (Figure 1).

Figure 1. Creating digital embryos using virtual morphogenesis (VM). Each embryo shown on right has a unique genotype. See text for details.

Each object in either scene in Figure 1 is a so-called 'digital embryo'. Digital embryos are naturalistic, virtual 3-D objects. They are created by simulating some of the key processes of embryonic development, such as morphogen-mediated cell division, cell growth, cell movement and programmed cell death. A given set of simulation parameters (or 'genotype') creates a unique, novel and naturalistic 3-D shape (Brady and Kersten, 2003; Hauffen et al., 2011). Since this creates shapes de novo by simulating natural morphogenetic processes, we refer to this as the Virtual Morphogenesis (VM) process.

The scenes in Figure 2 below were created by choosing many different digital embryos, applying a natural, 'leafy' surface texture to each embryo, and compositing selected embryos together to create camouflaged visual scenes. The two scenes have a single foreground object in common. However, because of the camouflage, this foreground object is hard to see, even though it is "in plain view", i.e., without occlusion, shadows, or other obfuscations.

Figure 2. Camouflaged visual scenes created using digital embryos. These two scenes have a single foreground object in common. Can you identify this object?

Just as one can create novel objects by simulating morphogenesis, one can create novel object categories by simulating the evolutionary process of natural selection (Hegdé, Bart and Kersten, 2008; Hauffen et al., 2011; also see Figure 3 below). We refer to this process as Virtual Phylogenesis (VM). Since VM and VP, respectively, allow the experimentor to create 'designer' objects and object categories with exquisite precision, we use these methodologies extensively in our research.

Figure 3. A digital embryo 'family tree'. This figure illustrates both VM and VP algorithms. The common ancestor is a simple icosahedron (top row). The the shapes of descendants reflect a combination of VM and VP, just as the shapes of biological objects reflect a combination of morphogenetic and phylogenetic processes. See text for details.

SELECTED REFERENCES

Bart, E., Hegdé, J. & Kersten, D. Fragment-based learning of visual categories. COSYNE 2008, 121 (2008).

Brady, M. J. & Kersten, D. Bootstrapped learning of novel objects. Journal of Vision 3, 413-422 (2003).

Hauffen, K., Bart, E., Brady, M., Kersten, D., Hegdé, J. Creating objects and object categories for studying perception and perceptual learning. Journal of Visualized Experiments (in press) (2011).

Hegdé, J., Bart, E., Kersten, D. Fragment-based learning of visual object categories. Current Biology 18:597-601 (2008).

Hegdé, J., Thompson, S. K., Brady, M. J. & Kersten, D. Object Recognition in Clutter: Cortical Responses Depend on the Type of Learning. NeuroImage (submitted) (2011).

Kromrey, S., Maestri, M., Hauffen, K., Bart, E. & Hegde, J. Fragment-based learning of visual object categories in non-human primates. PLoS One 5, e15444 (2010).

Shams, L., Brady, M. J., & Schaal, S. K. Graph matching vs. mutual information maximization for object detection. Neural Networks, 14, 345-354 (2001).

Vuong, Q. C. Visual categorization: when categories fall to pieces. Current Biology 18, R427-429 (2008).

Digital Embryos

A Brief History of Digital Embryos

Some Useful Links and Downloads

    You can create and manipulate your own digital embryos! To help you do this, we provide two different downloads below. The downloads are mutually independent, in that you download and use either one by itself. They are both provided here, mainly because they are written for two different platforms and because they have slightly different capabilities.

    Both are provided on a free, as-is basis. However, please do not hesitate to contact us if you encounter any problems.

    If you are a casual user, we recommend Download 1 below (Digital Embryo Workshop). If you want some of the more advanced capabilities of the digital embryo algorithms, use Download 2 below. Of course, you can use both downloads if you wish.

Download 1. Digital Embryo Workshop (DEW)

    This is a user-friendly, menu-driven program written by Dr. Mark Brady and Mr. Dan Gu. You can use this on a turn-key basis, in that you don't have to install anything else (such as libraries, drivers, etc) in order to use it.

    This version of DEW was specifically written for, and tested in, Windows XP. However, it also runs without any problems on all of the Windows 7 Ultimate (32-bit or 64-bit) machines we have tested to date.

    The installation instructions and some exemplar digital embryos are included in the download.   Operating instructions for DEW (i.e., instructions for creating and viewing digital embryos using DEW) can be found using the Help menu within DEW.

Download 2. Loose collection of Digital Embryo tools for Cygwin

    This download includes two executable (or .exe) files. They should be run within the Cygwin command interface in Windows. Cygwin is essentially a UNIX emulator for Windows. Cygwin itself is not included in our download. However, it can be downloaded free of charge from www.cygwin.org. For the purposes of running our tools, it is easiest to install the entire Cygwin package (i.e., with all the graphics libraries, etc). However, if you wish to do a custom installation, the sub-packages within Cygwin that you must install are listed in the 'Requisite_Cygwin_packages.txt' file included in this download.

    Once you have Cygwin running on your Windows machine, you can run our .exe files from the Cygwin command line by following the instructions in the 'Cygwin_CommandLine_Instructions.txt' file included in this download.

     The digital embryos created by the Cygwin executables are in an idiosyncratic, custom-written (albeit simple) file format called the .3dv file format. Viewing, rendering and/or manipulating digital embryos (including in the Digital Embryo Workshop) is far easier if it is in a more commonly used file format, such as the .obj format. We have included a Matlab script in this download (called 'Convert_3dv_to_obj.m') that converts any given .3dv file to a corresponding .obj file. Needless to say, this script should be run in Matlab. For the command-line syntax of this file, see the script itself, or type 'help Convert_3dv_to_obj' (without the quotes) at the Matlab prompt. Incidentally, this script can also be used to understand the .3dv file format.

    As you can probably tell, the Cygwin toolkit is not for the faint of heart at this time! We apologize for this. We plan to make this toolkit more user-friendly as our resources (by which we mostly mean our funds) allow.

Additional Information

    A short movie that shows the growth of a digital embryo can be viewed by clicking here. A bigger, a more colorful movie that shows another digital embryo grow can be downloaded from here. (Both these movies are included in Download 1 above.)

    For additional information about digital embryos, visit the digital embryo web page of Dr. Daniel Kersten of University of Minnesota. It is in his laboratory that digital embryos got their start.