Push the Red Button

As part of my ongoing attempts to create some nice datasets for training large code models for C/C++ , I've recently been attempting to build every package in Debian Unstable from source using bear to log the compilation and generate a compile_commands.json database for each build. Since it's not possible, in general, to parse C/C++ code without knowing what flags were used (e.g., so you can find header files, know what preprocessor defines are in use, etc.), this will open up some nice possibilities like: Getting ASTs for each source file Rebuilding each file and generating its LLVM IR (-emit-llvm) or assembly (-S) Extracting comments associated with individual functions I'll probably have more to say about this dataset once I actually get around to doing something fun with it, but for now I wanted to just jot down some notes on stuff I wish I had known before trying to do this: Isolation : Run the build for each package in some kind of isolated environment. You know how...

tl;dr : PANDA now supports detached replays (you don't need the underlying VM image to run a replay), and they can be shared at a new site called PANDA Share . Hooray for reproducibility! One of the most inspiring developments of the past few years has been the push for  open science , the movement to ensure that scientific publications, data, and software are freely available to all. In computer science, a big part of this has been a trend towards making software and experimental data available once a paper has been published, so that others can verify experiments and "stand on the shoulders of giants" by extending the software. There have also been  initiatives  aimed at making sure that the results of experiments in computer science can be replicated. In the latest release of PANDA, our Platform for Architecture-Neutral Dynamic Analysis, we've taken an important step in ensuring that experiments in dynamic analysis can be freely shared and replicated: as of  ...