MIT’s new technique could accurately simulate complex systems

Simulation is widely used in system design for evaluating different design options since testing ideas in the real world can be both costly and risky. However, it’s impossible to capture every detail of a complex system in a simulation. Therefore, trace-driven simulation is widely used technique that typically collects a small amount of real data that they replay while simulating the components they want to study.

Current trace-driven simulations assume that the interventions being simulated would not affect the validity of the traces. However, real-world traces are often biased by the choices algorithms make during trace collection.

To overcome this issue, a team of researchers from the Massachusetts Institute of Technology (WITH) has developed a new technique that eliminates this source of bias in trace-driven simulation.

Called CausalSim, the new system could enable unbiased trace-driven simulations. It could help researchers design better algorithms for a variety of applications, including improving video quality on the internet and increasing the performance of data processing systems.

CausalSim relaxes the exogenous trace assumption by explicitly modeling the fact that interventions can affect trace data. According to researchers, the new simulation method correctly predicted which newly designed algorithm would be best for video streaming compared to the existing simulators.

To simulate a new algorithm, CausalSim first estimates the latent factors at every time step of each trace. Then, it uses the estimated latent factors to predict the alternate evolution of the trace, actions, and observed variables of the component of interest, under the same latent conditions that were present when the trace was collected. This two-step process allows CausalSim to remove the bias in the trace data when simulating new algorithms.

Researchers used CausalSim to design an improved bitrate adaptation algorithm. It led them to select a new variant that had a stall rate – the amount of time a user spent rebuffering the video – that was nearly 1.4 times lower than a well-accepted competing algorithm, while achieving the same video quality.

On the other hand, an expert-designed trace-driven simulator predicted the opposite, indicating that this new variant should cause a stall rate that was nearly 1.3 times higher. The research team tested the algorithm on real-world video streaming and confirmed that CausalSim was correct.

“The gains we were getting in the new variant were very close to CausalSim’s prediction, while the expert simulator was way off. This is really exciting because this expert-designed simulator has been used in research for the past decade. If CausalSim can so clearly be better than this, who knows what we can do with it?” says Pouya Hamadanian, co-lead author of the paper.

Extensive evaluation of CausalSim on both real and synthetic datasets, including more than ten months of real data from the Puffer video streaming system, shows it improves simulation accuracy, reducing errors by 53% and 61% on average compared to expert-designed and supervised learning baselines.

In future work, the MIT team wants to apply CausalSim to situations where randomized control trial data are not available. They are also planning to explore how to design and monitor systems to make them more amenable to causal analysis.

Journal reference:

  1. Abdullah Alomar, Pouya Hamadanian, Arash Nasr-Esfahany, Anish Agarwal, Mohammad Alizadeh, and Devavrat Shah. CausalSim: A Causal Framework for Unbiased Trace-Driven Simulation. arXiv; DOI: 10.48550/arXiv.2201.01811

Note: This article have been indexed to our site. We do not claim legitimacy, ownership or copyright of any of the content above. To see the article at original source Click Here

Related Posts
Disney+ Yanks Content as It Raises Prices thumbnail

Disney+ Yanks Content as It Raises Prices

News @jdannychadwick May 14, 2023, 12:34 pm EDT | 1 min read Justin Duino / Review Geek On the heels of announcing a big price hike for Disney+, Disney has revealed its plans to remove certain content from the streaming platform and scale back new shows in production in a realignment of the company’s streaming
Read More
macOS Monterey: Update now or wait? thumbnail

macOS Monterey: Update now or wait?

Apple’s latest operating system for the Mac, macOS Monterey, is now available. And while Apple pushes users to upgrade right away, should you? There are a lot of good reasons for upgrading as soon as possible, but there are also a lot of good reasons to wait. If you’re trying to decide, let’s go over…
Read More
Blue Origin executive joins Voyager Space thumbnail

Blue Origin executive joins Voyager Space

by Jeff Foust — December 22, 2021 Clay Mowry (right) with Dylan Taylor, CEO of Voyager Space, after Taylor flew on Blue Origin’s New Shepard vehicle Dec. 11. Mowry is joining Voyager as its chief revenue officer. Credit: Twitter @claymowry WASHINGTON — Voyager Space has hired a Blue Origin executive as its chief revenue officer…
Read More
Naturally regrowing forests are helping to protect the remaining old forests in the Amazon thumbnail

Naturally regrowing forests are helping to protect the remaining old forests in the Amazon

Canopy of old-growth forest in the Peruvian Amazon. Credit: John Healey The climate crisis and UN Decade on Ecosystem Restoration have generated great interest in the value of secondary forests. These are forests that have regrown naturally on land abandoned from agriculture. Collaborative research between Lancaster University, Bangor University, and the University of British Columbia
Read More
Index Of News
Total
0
Share