Syntax-guided synthesis

2021

This paper presents PAYNT, a tool to automatically synthesise probabilistic programs. PAYNT enables the synthesis of finite-state probabilistic programs from a program sketch representing a finite family of program candidates. A tight interaction between inductive oracle-guided methods with state-of-the-art probabilistic model checking is at the heart of PAYNT. These oracle-guided methods effectively reason about all possible candidates and synthesise programs that meet a given specification formulated as a conjunction of temporal logic constraints and possibly including an optimising objective. We demonstrate the performance and usefulness of PAYNT using several case studies from different application domains; e.g., we find the optimal randomized protocol for network stabilisation among 3M potential programs within minutes, whereas alternative approaches would need days to do so.

Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 2020

We present a new method, Synthesis through Unification Genetic Programming (STUN GP), which synthesizes provably correct programs using a Divide and Conquer approach. This method first splits the input space by undergoing a discovery phase that uses Counterexample-Driven Genetic Programming (CDGP) to identify a set of programs that are provably correct under unknown unification constraints. The STUN GP method then computes these restraints by synthesizing predicates with CDGP that strictly map inputs to programs where the output will be correct. This method builds on previous work towards applying Genetic Programming (GP) to Syntax Guided Synthesis (Sy-Gus) problems that seek to synthesize programs adhering to a formal specification rather than a fixed set of input-output examples. We show that our method is more scalable than previous CDGP variants, solving several benchmarks from the SyGus Competition that cannot be solved by CDGP. STUN GP significantly cuts into the gap between GP and state-of-the-art SyGus solvers and further demonstrates Genetic Programming's potential for Program Synthesis.

Acta Informatica, 2017

Formal synthesis is the process of generating a program satisfying a high-level formal specification. In recent times, effective formal synthesis methods have been proposed based on the use of inductive learning. We refer to this class of methods that learn programs from examples as formal inductive synthesis. In this paper, we present a theoretical framework for formal inductive synthesis. We discuss how formal inductive synthesis differs from traditional machine learning. We then describe oracle-guided inductive synthesis (OGIS), a framework that captures a family of synthesizers that operate by iteratively querying an oracle. An instance of OGIS that has had much practical impact is counterexample-guided inductive synthesis (CEGIS). We present a theoretical characterization of CEGIS for learning any program that computes a recursive language. In particular, we analyze the relative power of CEGIS variants where the types of counterexamples generated by the oracle varies. We also consider the impact of bounded versus unbounded memory available to the learning algorithm. In the special case where the universe of candidate programs is finite, we relate the speed of convergence to the notion of teaching dimension studied in machine learning theory. Altogether, the results of the paper take a first step towards a theoretical foundation for the emerging field of formal inductive synthesis.

Software Engineering and Formal Methods, 2018

Code translation is a staple component of program analysis. A lifter is a code translation unit that translates low-level code to a higher-level intermediate representation (IR). Lifters thus enable a host of static and dynamic analyses for such low-level code. However, writing a lifter is a tedious manual process which must be repeated for every architecture an analysis aims to support. We introduce cross-architecture lifter synthesis, a novel approach that automatically synthesizes lifters for previously unsupported architectures. Our insight is that lifter synthesis can be bootstrapped with existing IR sketches that exploit the shared semantic properties of heterogeneous architecture instruction sets. We show that our approach automates a significant amount of translation effort for a previously unsupported instruction set, and that it enables discovery of new bugs on new architecture targets through reuse of an existing IR-based analysis.

ACM Transactions on Programming Languages and Systems

The success and popularity of deep learning is on the rise, partially due to powerful deep learning frameworks such as TensorFlow and PyTorch, which make it easier to develop deep learning models. However, these libraries also come with steep learning curves, since programming in these frameworks is quite different from traditional imperative programming with explicit loops and conditionals. In this work, we present a tool called TF-Coder for programming by example in TensorFlow. TF-Coder uses a bottom-up weighted enumerative search, with value-based pruning of equivalent expressions and flexible type- and value-based filtering to ensure that expressions adhere to various requirements imposed by the TensorFlow library. We train models to predict TensorFlow operations from features of the input and output tensors and natural language descriptions of tasks to prioritize relevant operations during search. TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding sim...

Artificial Intelligence and Industrial Applications, 2020

Databases store a vast amount of today's data and information, and to access that data users are required to have command over SQL or equivalent interface language. Hence, using a system that can convert a natural language to equivalent SQL query would make the data more accessible. In this sense, building natural language interfaces to relational databases is an important and challenging problem in natural language processing (NLP) and a widely studied field, and found recently momentum again due to the introduction of large-scale Datasets. In this paper, we present our approach based on word embedding and Recurrent Neural Networks (RNN), precisely on Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU) cells. We present also the DataSet used for training and testing our models, based on WikiSQL, and finally we show where we arrived in terms of accuracy.

Automated Reasoning, 2018

We introduce a new theory of algebraic datatypes where selector symbols can be shared between multiple constructors, thereby reducing the number of terms considered by current SMT-based solving approaches. We show that the satisfiability problem for the traditional theory of algebraic datatypes can be reduced to problems where selectors are mapped to shared symbols based on a transformation provided in this paper. The use of shared selectors addresses a key bottleneck for an SMT-based enumerative approach to the Syntax-Guided Synthesis (SyGuS) problem. Our experimental evaluation of an implementation of the new theory in the SMT solver 4 on syntax-guided synthesis and other domains provides evidence that the use of shared selectors improves state-of-the-art SMT-based approaches for constraints over algebraic datatypes.

Logic for Programming, Artificial Intelligence, and Reasoning, 2015

In this paper, we propose a unified framework for designing static analysers based on program synthesis. For this purpose, we identify a fragment of second-order logic with restricted quantification that is expressive enough to capture numerous static analysis problems (e.g. safety proving, bug finding, termination and non-termination proving, superoptimisation). We call this fragment the synthesis fragment. We build a decision procedure for the synthesis fragment over finite domains in the form of a program synthesiser. Given our initial motivation to solve static analysis problems, this synthesiser is specialised for such analyses. Our experimental results show that, on benchmarks capturing static analysis problems, our program synthesiser compares positively with other general purpose synthesisers.

ACM SIGPLAN Notices, 2016

Inductive invariants can be robustly synthesized using a learning model where the teacher is a program verifier who instructs the learner through concrete program configurations, classified as positive, negative, and implications. We propose the first learning algorithms in this model with implication counter-examples that are based on machine learning techniques. In particular, we extend classical decision-tree learning algorithms in machine learning to handle implication samples, building new scalable ways to construct small decision trees using statistical measures. We also develop a decision-tree learning algorithm in this model that is guaranteed to converge to the right concept (invariant) if one exists. We implement the learners and an appropriate teacher, and show that the resulting invariant synthesis is efficient and convergent for a large suite of programs.

Companion Proceedings of the 2021 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity, 2021

Program synthesis has seen many new applications in recent years, in large part thanks to the introduction of SyGuS. However, no existing SyGuS solvers have support for synthesizing recursive functions. We introduce an multi-phase algorithm for the synthesis of recursive "looplike" programs in SyGuS for programming-by-example. We solve constraints individually and treat them as "unrolled" examples of how a recursive program would behave, and solve for the generalized recursive solution. Our approach is modular and supports any SyGuS Solver.

Scientific Reports, 2022

This work describes an approach for synthesizing state-feedback controllers for discrete-time systems, taking into account performance aspects. The proposed methodology is based on counterexampleguided inductive synthesis (CEGIS), producing safe controllers based on step response performance requirements, such as settling time and maximum-overshoot. Controller candidates are generated through constrained optimization based on genetic algorithms. Each iteration that does not satisfy the initial system requirements is learned as a failed result and then used in another attempt. During the verification phase, it is considered the controller fragility to ensure deployable implementations. Such an approach assists the discrete-time control system design since weaknesses occur during implementation on digital platforms, where systems that meet design requirements are employed. The proposed method is implemented in DSVerifier, a tool that uses bounded (and unbounded) model checking based on satisfiability modulo theories. Experimental results showed that our approach is practical and sound regarding the synthesis of discrete state-feedback control systems that present performance requirements. It considers finite word-length effects, unlike other methods that routinely ignore them. The design of digital controllers is one of the essential tasks regarding digital control systems 1. In particular, such structures consist of sensors, controlled system, control algorithms, and actuators, which together seek to maintain the desired behavior of plants' (controlled system) variables under control, i.e., they ensure the desired transient and steady-state responses 2. The digital control theory aims to preserve some properties, based on discrete-time models, e.g., stability and robustness. Those properties are necessary for the correct operation of real plants through a digital controller; however, controlling continuous systems with digital controllers raises new challenges such as: round-off and quantization of samples and coefficients in digital controllers, due to finite word-length (FWL) implementations, can lead to overflow, poles and zeros sensitivity, limit-cycle oscillation (LCO), approximation errors, and other types of deviations from expected behavior, which might cause system instability and performance degradation 3,4. Indeed, the sensitivity concerning implementation issues is called fragility 3 and the design of control systems should address that issue, which drives the attention of control systems and verification communities. Some initiatives applying formal verification to dynamic control systems have been developed 4-8 in the past years which includes verification methods that take into account implementation aspects and determine uncertain linear-system stability regarding digital controllers 4 , schemes that address validation of robustness at both model and code level 5 , approaches for verifying digital filters w.r.t overflow and approximation errors, due to quantization and round-off effects 6 , and controller synthesis, in an attempt to generate correct-by-construction elements 8. Moreover, there exists also an approach that aims to formally verify whether a digital control system meets performance requirements, such as settling time and maximum overshoot, using both open-and closedloop forms and considering FWL effects 9. Such a work gathered on a unified framework, fragility aspects, and performance requirements, making it possible to check systems and ensure that they could be promptly deployed on real scenarios.

Computational Methods in Systems Biology, 2018

Vesicle Traffic Systems (VTSs) are the material transport mechanisms among the compartments inside the biological cells. The compartments are viewed as nodes that are labeled with the containing chemicals and the transport channels are similarly viewed as labeled edges between the nodes. Understanding VTSs is an ongoing area of research and for many cells they are partially known. For example, there may be undiscovered edges, nodes, or their labels in a VTS of a cell. It has been speculated that there are properties that the VTSs must satisfy. For example, stability, i.e., every chemical that is leaving a compartment comes back. Many synthesis questions may arise in this scenario, where we want to complete a partially known VTS under a given property. In the paper, we present novel encodings of the above questions into the QBF (quantified Boolean formula) satisfiability problems. We have implemented the encodings in a highly configurable tool and applied to a couple of found-in-nature VTSs and several synthetic graphs. Our results demonstrate that our method can scale up to the graphs of interest.

Computer Aided Verification, 2020

We propose a general end-to-end deep learning framework Code2Inv, which takes a verification task and a proof checker as input, and automatically learns a valid proof for the verification task by interacting with the given checker. Code2Inv is parameterized with an embedding module and a grammar: the former encodes the verification task into numeric vectors while the latter describes the format of solutions Code2Inv should produce. We demonstrate the flexibility of Code2Inv by means of two small-scale yet expressive instances: a loop invariant synthesizer for C programs, and a Constrained Horn Clause (CHC) solver.

ACM Transactions on Architecture and Code Optimization

This article presents a code generator for sparse tensor contraction computations. It leverages a mathematical representation of loop nest computations in the sparse polyhedral framework (SPF), which extends the polyhedral model to support non-affine computations, such as those that arise in sparse tensors. SPF is extended to perform layout specification, optimization, and code generation of sparse tensor code: (1) We develop a polyhedral layout specification that decouples iteration spaces for layout and computation; and (2) we develop efficient co-iteration of sparse tensors by combining polyhedra scanning over the layout of one sparse tensor with the synthesis of code to find corresponding elements in other tensors through an SMT solver. We compare the generated code with that produced by a state-of-the-art tensor compiler, TACO. We achieve on average 1.63× faster parallel performance than TACO on sparse-sparse co-iteration and describe how to improve that to 2.72× average speedu...

Proceedings of the ACM on Programming Languages, 2020

While editing code, it is common for developers to make multiple related repeated edits that are all instances of a more general program transformation. Since this process can be tedious and error-prone, we study the problem of automatically learning program transformations from past edits, which can then be used to predict future edits. We take a novel view of the problem as a semi-supervised learning problem: apart from the concrete edits that are instances of the general transformation, the learning procedure also exploits access to additional inputs (program subtrees) that are marked as positive or negative depending on whether the transformation applies on those inputs. We present a procedure to solve the semi-supervised transformation learning problem using anti-unification and programming-by-example synthesis technology. To eliminate reliance on access to marked additional inputs, we generalize the semi-supervised learning procedure to a feedback-driven procedure that also ge...

Communications of the ACM, 2018

A promising, useful tool for future programming development environments.