Sybl

Open simulation: new practices in AI research, urbanism, and cyber-physical systems

In recent decades, architects, designers, scientists, and engineers have turned to computer simulation in the hope of creating more functional, sustainable and robust designs. Computer-aided design has been applied not only to the design of individual elements but also to large complex systems such as factories, entire cities, robots, and autonomous vehicles. A variety of simulation tools have been developed to model specific aspects of complex phenomena such as thermodynamics, climate, crowd behavior, structural integrity, economic and financial behavior. Nevertheless, numerous challenges remain. For example, although many factors that influence the design of complex systems are interdependent, they are often analyzed in isolation due to the lack of interoperable formats that allow different simulation engines to be linked together. A systems approach that combines visual programming, state-of-the-art modeling, and open simulation architecture can help data scientists, AI researchers, and engineers combine their expertise to create integrated models that better capture the complexities found in the real world. In this short article, we describe Sybl: an open and modular simulation environment designed to (1) simplify the modeling of complex systems (2) facilitate the integration of different simulation engines and (3) enable interactions across different scales and domains.

Discipline: Data visualizaiton, CAD, data science
Role: research, interface and software design
Team: Mark Wilcox, Mariia Fedorova
Research Partner: Strelka Media Institute
Year: 2019

Introduction

Simulations are based on the process of modeling a real phenomenon with a series of mathematical formulas. A computer program that allows a user to observe an operation without actually performing it. Besides emulating processes, simulations are also used in scientific research to test new theories. Once a theory of causal relationships has been established, the relationships can be codified in the form of a computer program. The results can be correlated with phenomena from the real world to compare whether the assumptions in the model were "good enough". If the program behaves in the same way, there is a good chance that the proposed relationships are correct.

Advanced computer programs can simulate the behavior of energy systems, weather conditions, electronic circuits, chemical reactions, control systems, atomic reactions, and even complex biological processes. In theory, all phenomena that can be reduced to mathematical data and equations can be simulated on a computer. Modeling physical phenomena can be difficult since most natural phenomena are subject to an almost infinite number of influences, which are often difficult to observe.

Simulations can better model complex systems that are untenable for analytical calculations or too distant or dangerous for experimental verification. Because of their value at the beginning of the design process and during testing, designers rely on simulation throughout the production process to reduce the cost of iterative prototyping. Although the initial setup of simulation environments requires considerable effort, the benefits outweigh the challenges in robotics, system control, and various applications -- especially during the design and deployment of equipment where testing capabilities on live devices are not possible, safe, or readily available. For example, when the penalty for improper operation is costly, such as for aircraft pilots and nuclear power plant operators, a mock-up of the actual control panel is combined with a real-time simulation of the physical reaction, providing valuable training experience without fear of a catastrophic outcome.

The Digital Twin

“In the past, factory managers had their office overlooking the factory so that they could get a feel for what was happening on the factory floor. With the digital twin, not only the factory manager, but everyone associated with factory production could have that same virtual window to not only a single factory, but to all the factories across the globe.” (Grieves, 2014, p. 5)

A digital twin is essentially a digital replica of physical assets, processes, and systems within a limited area, such as a warehouse, town or mine. The twin provides a representation of both the core elements and dynamics of IoT devices used in the space and system represented. Digital twins use AI, machine learning and software analytics to create real-time digital simulation models that can be updated and modified as their real, physical counterparts or "twins" change. A digital twin applied to a city would look like a real-time "video game" version of urban space.

Problems with current simulation environments

There are about 400 different software packages that perform different types of computer modeling tasks. Despite their wide availability, most software applications are limited in the functions they support and don't complement each other. Despite major advances in digital modeling, rendering engines, and AI methods, the current architecture of most simulation tools is not capable of taking into account the dynamics of an increasingly complex and interconnected world.

The growing popularity of game engines as a simulation platform for autonomous vehicles points to the necessity for an easy to deploy and modular environment that enables researchers to quickly add functionalities as their models grow over time. While offering designers valuable features, Unity's app store doesn't support more formats and real-time data necessary to increase the model's accuracy for real-world applications such as robotics, and autonomous vehicles (e.g. live traffic conditions, map data, pedestrian activity in the city). While offering a high degree of modularity, game engines are not built on a universal standard, which makes it incompatible with existing simulation software. Such limitations benefit large research groups and pose serious challenges for smaller research teams that cannot afford the development of custom software.

Responsive image

Introducing Sybl

1. Interoperability, and modularity

Sybl extends popular programming and simulation engines through a visual programming language in which simulation models can be combined, increasing their resolution, accuracy and dynamic complexity. The system extends the current paradigm of stand-alone software architecture by bridging different simulation packages, establishing a link between different engines and their disciplines. The dynamic complexity that results from such an environment gives designers a unique perspective for modeling complex systems and applications.

2. Simulations and synthetic datasets for AI

It is expected that synthetic datasets for AI will enable the next breakthrough in AI research. Due to the high fidelity found in physical engines, game environments such as Unity and Unreal are widely used by AI researchers to train and validate algorithms for computer vision in a secure and controlled environment. The higher the accuracy, the easier it is to transfer learnings from simulations to the real world. Sybl's modular design allows researchers to create complex environments that closely match real-world phenomena, from accurate physics to complex network of motor actuators used in robotics.

3. Co-generative computer-aided-design

Simulations in Sybl run in an open environment, allowing different experts to collaborate across organizations and domains in real-time. This model enables insurance companies, urbanists, and AV engineers to simulate different aspects of tomorrow's city, as well as the legal and physical infrastructure required. The impact of small changes in one model can be tested in real-time by all stakeholders involved.

Conclusion

Instruments not only determine what can be done: they also determine, to some degree, what can be thought. The way we think and judge the world is shaped by the instruments available to us at any given time. Simulations, machine learning, and the digital twin are the predominant tools with which we shape the future of our physical and digital spaces. Processes of data acquisition, organization and modeling eventually become the main window from which we make sense of the world. Simulations not only function as a representation of the underlying processes, they often exert a recursive force that acts as a kind of read/write medium. This effect is visible as the digital becomes the constant object, while the physical is in a constant state of update and optimization. It's extremely important for the data science, HCI and machine learning community to critically rethink the underlying architectures that power most modeling environments today, as to both address its shortcomings as well as explore more open and interconnected designs. We hope that the design and research of Sybl can shed light into the future of CAD, simulations and complexity science more broadly.

Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image
Responsive image