Search-based program synthesis

Artificial Intelligence, 1983

Efficiency is a problem in automatic programming-both in the programs produced and in the synthesis process itself. The efficiency problem arises because many target-language programs (which vary in their time and space performance) typically satisfy one abstract specification. This paper presents a framework for using analysis and searching knowledge to guide program synthesis in a stepwise refinement paradigm. A particular implementation of the framework, called LIBRA, is described. Given a program specification that includes size and frequency notes, the performance measure to be minimized, and some limits on synthesis resources, LIBRA selects algorithms and data representations and decides whether to use 'optimizing' transformations. By applying incremental, algebraic program analysis, explicit rules about plausible implementations, and resource allocation on the basis of decision importance, LIBRA has guided the automatic implementation of a number of programs in the domain of symbolic processing.

arXiv (Cornell University), 2023

This paper presents Squid, a new conjunctive query synthesis algorithm for searching code with target patterns. Given positive and negative examples along with a natural language description, Squid analyzes the relations derived from the examples by a Datalog-based program analyzer and synthesizes a conjunctive query expressing the search intent. The synthesized query can be further used to search for desired grammatical constructs in the editor. To achieve high efficiency, we prune the huge search space by removing unnecessary relations and enumerating query candidates via refinement. We also introduce two quantitative metrics for query prioritization to select the queries from multiple candidates, yielding desired queries for code search. We have evaluated Squid on over thirty code search tasks. It is shown that Squid successfully synthesizes the conjunctive queries for all the tasks, taking only 2.56 seconds on average. 2012 ACM Subject Classification Software and its engineering → Automatic programming; Humancentered computing → User interface programming

ACM Transactions on Programming Languages and Systems, 2018

In this article, we propose a unified framework for designing static analysers based on program synthesis . For this purpose, we identify a fragment of second-order logic with restricted quantification that is expressive enough to model numerous static analysis problems (e.g., safety proving, bug finding, termination and non-termination proving, refactoring). As our focus is on programs that use bit-vectors, we build a decision procedure for this fragment over finite domains in the form of a program synthesiser. We provide instantiations of our framework for solving a diverse range of program verification tasks such as termination, non-termination, safety and bug finding, superoptimisation, and refactoring. Our experimental results show that our program synthesiser compares positively with specialised tools in each area as well as with general-purpose synthesisers.

Information Processing Letters, 2019

Program synthesis aims to mechanize the task of programming from the user intent (expressed in various forms like pre/post conditions, examples, sketches, etc). There are many approaches to program synthesis that are usually implemented in isolation: deductive, syntax-based, inductive, etc. In this paper, we describe PSMF2, a program synthesizer that combines model finder and genetic programming. PSMF2 takes as user intent examples and a soft sketch: a new kind of user intent defined as a set of commands that must appear in the synthesized program (and that are in no particular order of execution). The output of PSMF2 is a general purpose imperative program. The combination of inductive synthesis and genetic programming has allowed PSMF2 to synthesize 7 programs (IntSQRT, Majority of 5, Majority of 8, Max of 4, Modulo, Factorial, and Fibonacci) found in the SyGuS competition, the iJava and IntroClass, and the Genetic programming communities. We carried out an empirical evaluation on the synthesis time of these 7 programs and the mean time varied from 56.4 seconds (Majority of 5) to 15.9 minutes (Fibonacci).

Proceedings of the companion publication of the 13th international conference on Modularity - MODULARITY '14, 2014

Program generators and transformations are hard to implement correctly, because the implementation needs to generically describe how to construct programs, for example, using templates or rewrite rules. We apply dynamic analysis to program generators in order to support developers in finding bugs and identifying the source of the bug. Our analysis focuses on syntactic language constraints and checks that generated programs are syntactically well-formed. To retain a language's grammar as the unique specification of the language's syntax, we devised mechanisms to derive the analysis from the grammar. Moreover, we designed a run-time system to support the modular activation/deactivation of the analysis, so that generators do not require adaption. We have implemented the analysis for the Stratego term-rewriting language and applied it in case studies based on Spoofax and SugarJ.

Cornell University - arXiv, 2022

We present Scrybe, an example-based synthesis tool for a statically-typed functional programming language, which combines topdown deductive reasoning in the style of λ 2 with Smyth-style live bidirectional evaluation. During synthesis, example constraints are propagated through sketches to prune and guide the search. This enables Scrybe to make more effective use of functions provided in the context. To evaluate our tool, it is run on the combined, largely disjoint, benchmarks of λ 2 and Myth. Scrybe is able to synthesize most of the combined benchmark tasks.

Proceedings 12th IEEE International Conference Automated Software Engineering, 1997

Proof search for the structural synthesis of programs (SSP) -a deductive program synthesis method which is suited for compositional programming in large and is in practical use in a number of programming environments is explained. SSP is based on a decidable logical calculus where complexity of the proof search is still PSPACE. This requires paying special attention to the efficiency of search. The practice of application of SSP has given its several modifications and extensions. Besides the general case of SSP and its strategies, we present synthesis with independent subtasks, a number of heuristics used for speeding up the search and partial deduction in the framework of SSP.

Logic for Programming, Artificial Intelligence, and Reasoning, 2015

In this paper, we propose a unified framework for designing static analysers based on program synthesis. For this purpose, we identify a fragment of second-order logic with restricted quantification that is expressive enough to capture numerous static analysis problems (e.g. safety proving, bug finding, termination and non-termination proving, superoptimisation). We call this fragment the synthesis fragment. We build a decision procedure for the synthesis fragment over finite domains in the form of a program synthesiser. Given our initial motivation to solve static analysis problems, this synthesiser is specialised for such analyses. Our experimental results show that, on benchmarks capturing static analysis problems, our program synthesiser compares positively with other general purpose synthesisers.

Cornell University - arXiv, 2021

Program synthesis is challenging largely because of the difficulty of search in a large space of programs. Human programmers routinely tackle the task of writing complex programs by writing sub-programs and then analyzing their intermediate results to compose them in appropriate ways. Motivated by this intuition, we present a new synthesis approach that leverages learning to guide a bottom-up search over programs. In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a given set of input-output examples. This is a powerful combination because of several emergent properties. First, in bottom-up search, intermediate programs can be executed, providing semantic information to the neural network. Second, given the concrete values from those executions, we can exploit rich features based on recent work on property signatures. Finally, bottom-up search allows the system substantial flexibility in what order to generate the solution, allowing the synthesizer to build up a program from multiple smaller sub-programs. Overall, our empirical evaluation finds that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches. We demonstrate the effectiveness of our technique on two datasets, one from the SyGuS competition and one of our own creation.

Proceedings of the Genetic and Evolutionary Computation Conference Companion

Program synthesis techniques offer the potential to allow nonprogrammers to create computer programs. Approaches based on evolutionary algorithms are known for their performance in program synthesis. Since 2015, the general program synthesis benchmark suite offers a common pool of benchmark problems, which makes different approaches comparable. To analyze what has been achieved so far, we identified in a recent literature review the main evolutionary program synthesis approaches, determined the difficulty of the common benchmark problems, and discussed current challenges. In this short paper, we present the difficulty ranking of the benchmark problems and summarize the current challenges in program synthesis with evolutionary algorithms. CCS CONCEPTS • Theory of computation → Evolutionary algorithms; • Software and its engineering → Search-based software engineering.