Deep Learning with 3D Virtual Environments

No reality, No problem.

Keno Leon
9 min readMar 20, 2024

--

As part of my research into experiential AIs — that is, AIs that learn by experiencing things on their own, much like we do — I recently started working on Virtual 3D environments for AIs. These environments offer a safe sandbox for AI development. Given the complexity and novelty of the topic, I began with AIs on the simpler (algorithmic) side, mainly focusing on search processes.

🖐️ Pardon the interruption but I am testing out a Substack publication solely focused on practical and theoretical AIs :

You can find it here : The AI Garden , Thanks ! 🖐️

This is a first look into bridging the gap between more advanced and current AIs, incorporating Deep Learning workflows and tools like Keras/Tensorflow.

The task/experiment is somewhat straightforward: set up a 3D virtual environment, introduce an AI/Agent, incorporate an object, and initiate training for object recognition (initially focusing on a single object). If this seems a bit daunting, it is! Primarily because there are several layers and technologies we need to navigate, let’s break it down:

--

--