Industrial microservice architectures vary so wildly in their characteristics, such as size or communication method, that comparing systems is difficult and often leads to confusion and misinterpretation. In contrast, the academic testbeds used to conduct microservices research employ a very constrained set of design choices. This lack of systemization in these key design choices when developing microservice architectures has led to uncertainty over how to use experiments from testbeds to inform practical deployments and indeed whether this should be done at all. We conduct semi-structured interviews with industry participants to understand the representativeness of existing testbeds’ design choices. Surprising results included the presence of cycles in industry deployments, as well as a lack of clarity about the presence of hierarchies. We then systematize the possible design choices we learned about from the interviews, and identify important mismatches between our interview results and testbeds’ designs that will inform future, more representative testbeds.

Identifying mismatches between microservice testbeds and industrial perceptions of microservices.

There is no standard approach to compare the success of different neural network architectures utilized for time series synthesis.
This hinders the evaluation and decision, which architecture should be leveraged for an unknown data set.
We propose a combination of metrics, which empirically evaluate the performance of neural network architectures trained for time series synthesis.
With these measurements we are able to account for temporal correlations, spatial correlations and mode collapse issues within the generated time series.

We further investigate the interaction of different generator and discriminator architectures between each other.
The considered architectures include recurrent neural networks, temporal convolutional networks and transformer-based networks.
So far, the application of transformer-based models is limited for time series synthesis.
Hence, we propose a new transformer-based architecture, which is able to  synthesise time series.
We evaluate the proposed architectures and their comobinations in over 500 experiments, amounting to over 2500 computing hours.
We provide results for four data data sets, one univariate and three multivariate.
The data sets vary with regard to length, patterns in temporal and spatial correlations.
One of the multivariate data sets is an artificial data set constructed in a conditional setup.

We use our metrics to compare the performance of generative adversarial network architectures for time series synthesis.
To verify our findings we utilize quantitative and qualitative evaluations.
Our results indicate that temporal convolutional networks outperform recurrent neural network and transformer based approaches with regard to fidelity and flexibility of the generated data.
Temporal convolutional network architecture are the most stable architecture for a mode collapse prone data set.
The performance of the transformer models strongly depends on the data set characteristics, it struggled to synthesise data sets with high temporal and spatial correlations.
Discriminators with recurrent network architectures suffered immensely from vanishing gradients.
We also show, that the performance of the generative adversarial networks depends more on the discriminator part rather than the generator part.

SoK: The Great GAN Bake Off, An Extensive Systematic Evaluation of Generative Adversarial Network Architectures for Time Series Synthesis

Patching vulnerabilities in third party libraries is critical for maintaining security, yet such patches can take over 500 days to be distributed on average. Manually creating binary patches requires semantic analysis to identify the full set of functions present in the library. Existing semantic binary analysis approaches do not scale, or are inaccurate. 

In this paper, we introduce IOVec Function Identification (IOVFI), which assesses similarity based on program state transformations,
which compilers largely guarantee even across compilation environments and architectures. IOVFI executes functions with initial predetermined program states, measures the resulting program state changes, and uses the sets of input and output state vectors as unique semantic fingerprints. Since IOVFI relies on state vectors, and not code measurements, it withstands broad changes in compiler, optimization, underlying architecture, and even different implementations of equivalent functionality. Crucially, IOVFI is the first approach
to support architecture independent classification.  Evaluating our IOVFI implementation as a semantic function identifier for coreutils-8.32, we achieve a high .779 average F-Score, indicating high precision and recall. When identifying functions generated from differing compilation environments, IOVFI achieves a 101% accuracy improvement over the most-recent BinDiff 6, outperforms asm2vec in cross-
compilation environment accuracy, and, when compared to dynamic frameworks, BLEX and IMF-SIM, IOVFI is 25%–53% more accurate. Additionally, we show that IOVFI is largely unaffected by code obfuscation by achieving similarly high accuracy against obfuscated code. To demonstrate that state transformations are capable of cross-ISA identification, IOVFI achieves similarly high accuracy rates when identifying AArch64 functions using unmodified x64 classification vectors. We show that IOVFI scales to large binaries by evaluating semantic identification accuracy for three large and commonly used libraries: libxml2, libpng, and libz. Finally, we perform a semantic history analysis of libpng and libz on 14 different versions. We correctly identify libpng versions distributed with the last five years of Ubuntu releases.

JSYS 2022 Feb Papers Submissions

Identifying mismatches between microservice testbeds and industrial perceptions of microservices.

SoK: The Great GAN Bake Off, An Extensive Systematic Evaluation of Generative Adversarial Network Architectures for Time Series Synthesis

Accurate Compiler, Optimization, and Architecture Independent Function Identification using Program State Transformations